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ABSTRACT 

The Sullivan Reading Program is designed to teach 
reading, with the focus on basic skills, to children beginning in 
kindergarten and continuing through grade 3, It is also used in 
reTOdial reading instruction for children and adults of all ages. The 
two basic series of materials are the Programmed Reading series and 
the Sullivan Reading Program, accompanied by many supplementary 
materials. This report describes the rationale, various materials 
developed and the procedures for using them, the origins and 
developmental procedures, and the field tests carried out by 
independent school districts and researchers. It is based upon 
published materials, documents in the files of the developing agency, 
and interviews with staff who were Involved in the product 
devalopment. Diffusion, adoption, costs, and critical decisions made 
during the development are listed. This product development report is 
one of 21 such reports, each dealing with the developmental history 
of a recent educational product. References, tables, and appendixes 
are included- (AW) 
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PREPACE 



This product development report is one of 21 such reports j each dealing 
with the developmental history of a recent educational product. A list of the 
21 products, and the agencies responsible xor their development, is contained 
in Appendix p to this report. The study, of which this report is a component, 
was supported by U.S* Office of Education Contract No. OEC-0^70-4892 , entitled 
The Evaluation of the Impact of Educational Research and Development Products-” 
The overall project was designed to examine the process of development of 
successful educational products-” 

Tnia report represents a relatively unique attempt to document what 
occurred in the development of a recent educational product that appears to 
have potential impact. The report is based upon published materials, docu- 
ments in ae files of the developing agency, and interviews with staff who 
were involved in the development of the product* A draft of each study was 
Leviewed by the developer’s staff* Generally, their suggestions for revisions 
were tncorporated into the text; however, complete responsibility for inter= 
pretations concerning any facet of development, evaluation, and diffusion 
rests with the authors of this report. 

Although awareness of the all impact of the study requires reading both 
the individual product development reports and the separate final report, each 
study may be ^.ead individually. For a quick overview of essential events in 
the product history , the reader is referred to those sections of the report 
containing the flow chart and the critical decision record. 

Ihe final report contains s a complete discussion of the procedures and 
the selection criteria used to identify exemplary educational products; gener- 
alizations drawn from the 21 product devalopment case studies; a comparison of 
these generalizations with hypotheses currently existing in the literature 
regarding the processes of innovation and change; and the identification of 
some proposed data sources through which the U.S. Office of Education could 
monitor the impact of developing products. The final report also Includes a 
detailed outline of the search procedures and the information sought for each 
case report. 

Permanent project staff consisted of Calvin E, Wright, Principal 
Investigator; Jack J, Crawford, Project Director; Daniel W. Kratochvil, Research 
Scientist; and Carolyn A, Morrow, Administrative Assistant. In addition, other 
staff who assisted in the preparation of individual product reports are identi- 
fied on the appropriate title pages. The Project Monitor was Dr. Alice Y. 

Scares of the USOE Office of Program Planning and Evaluation, 

Sincere gratitude is extended to those overburdened staff members of the 
2i product development studies who courteously and freely gave their time so 
that we might present a detailed and relatively accurate picture of the events 
in the development of some exemplary educational research and development pro- 
ducts. If we have chronicled a just and moderately complete account of the 
birth of these products dnd the hard work that spawned them, credit lies with 
those staff members of each product development team who ransacked memory and 
files to recreate history. 
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PR0L3UCT DESCFlIPTION 



Produce Characteristics 



Name 

The Sullivan Reading Program. 



Sullivan Associates j 



Menlo Park , 



California . 



Distributo r 

Behavioral Research Laboratories, Inc., (BRL) Palo Alto, California. 
Research, development, and publication have all been accomplished by the 
same group of people; only their locations and professional affiliations 
have changed. BRL markets most of the reading materials, although some are 
distributed by the Webster Division of the McGraw“Hill Book Company. 

Focus 

The Sullivan Reading Program focuses on the basic skills of reading 
the English language - 

Grade Level 

Kindergarten-grade 3 primarily, but see Target Population, below. 
Target Population 

Designed to teach reading to all youngsters beginning in kindergarten 
or first grade and continuing through the primary years, the Sullivan 
materials arc also used in remedial reading instruction for children and 
adults of all ages. 



Rationale for Product 

The Sullivan Reading Program is the work of Dr. M. W. Sullivan, a few 
of his colleagues s hundreds of assistants, and thousands of children who 
wanted to learn to read. Dr. Sullivan strives to teach all children to 
read, to read well, and to read with enjoyment and satisfaction. Hia 
materials approach this task through linguistics, development of decoding 
skills, and a prograiraned format that both stimulates and reinforces the 
child in his efforts to read. Applying the tenets of both learning theory 
and linguistics, Sullivan concluded that standard methods of teaching 
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reading were making the subject much more difficult than it needed to be. 
Accepted learning theory states that learning is accomplished by the 
responses of the pupil to what is presented, not in the mere reception of 
information. Secondly, learning theory states that for learning to be 
efficient, a pupil's response must be immediately corrected if the response 
is wrong, or immediately reinforced or rewarded if it is right. A third 
principle is that all pupils dc not learn at the same rate; to maximize 
learning for each student, each should be allowed to progress at his own 
rate rather than be prodded or kept in check to work at the same pace as 
the group as a whole • The field of linguistics provides a completely new 
schedule for teaching the sounds and words of English—a schedule designed 
never to fool or trick the child by the early presentation of words having 
few or no phonetic analogues. 

Description of Materials 

The Sullivan Reading Program actually consists of several different 
series of materials, each with a different name, and different purpose. The 
backbone of the materials is made up of two basic series, the Progranmied 
Reading series and the Sullivan Reading Program, Many supplementary sets 
of materials, including reading readiness materials, enrichment activities, 
and tapes have been produced to accompany and enhance the two basic series. 
The materials may be used and purchased in any combination, depending upon 
user needs. At this time, the niimber and variety of Sullivan reading 
materials are tncreasing at a steady pace; consequently, there are compon^ 
ents of the Sullivan program that are not discussed here. The major products 
under consideration in this report are described below. 

Readiness in Language Arts Program , by C, D, Buchanan and M. W. Sullivan, 
is entirely teacher administerad • This program prepares children for the 
formal reading progr^ by teaching directions, spatial concepts, colors, the 
alphabet, and one sound-symbol relationship for each letter. Materials 
include six Giant Books with color pictures, six teacher *s manuals giving 
step^by"*step directions, two alphabet strips (small letters and capitals), 
and an easel and case to present and pack the materials. The cost of this 
program is $199,95, 

Enrichment Materials Kit for Readiness in Language Arts , by C, D, 

Buchanan and M, W* Sullivan, is designed to accompany the program above. 
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Thase matei'iials allow the children to use and review what they learn In the 
Readiness Program. The kit includes a teacher's manual, three different 
colorbooks, 90 enrichment cards, a set of letter cards, and three sets of 
Word cards. The colorbooks allow the children to use actively the color 
discrimination presented in the Giant Books; the enricliment cards are used 
to initiate discussion of the concepts presented; and the letter and word 
cards provide oral review of lessons in the Readiness Program. A complete 
set for 30 pupils costs $69.95. 

Ihe I Can Read series , by Sullivan Associates, consists of eight readers 
that pupils are capable of reading alone, during and inmiediately after the 
Readiness Program, The complete classroom set of five copies of each of the 
eight readers costs $49.95. 

Reading Readiness prepares pupils for entry into the Sullivan Reading 
Program by teaching printed numbers and letters, sound-synibol relationships, 
and a few words. Book A is teacher administered, Book B reviews Book A 
concepts and gives the child his first opportunity to work individually at 
his own speed; Book C presents review and new letters and words; and Book D 
introduces syntactic patterns (e.g. , article-noun pattern) and teaches new 
letters and words. Six Reading Readiness Readers supplement the program 
by providing enjoyable reading the child can master. A Teacher’s Manual 
and Placement Examination are included in the program which costs $16.38 
for one of each. 

The Sullivan Decoding Kit is designed specifically for the first grade 
pupil and can be used to augment the basal reading series. Word cards show 
on one side a picture and its corresponding noun and on the other side the 
noun alone. One copy each of Books A, B, C, and D from the program above 
and their accompanying teacher's manual are included. An alphabet chart 
and a second teacher's manual giving step— by—step instruction for use of 
the kit are included. Materials for 30 pupils cost $199.95, and a single 
kit is priced at $49.95. 

The Sullivan Reading Program , by M. W. Sullivan, Is designed to help 
children who have reading problems. The program is divided in five numbered 
series, each corresponding roughly to one school year. Series 1 and Series 2 
each consist of four progranmied texts ($1,49 each) wherein the child experi- 
ences continuous success in small, easy tasks, is reinforced constantly for 




correct answers , and proceeds easily from known information to new informa“ 
tlon. Twelve readers ($*99 each) accompany the texts in each series. 

Series 3 has four texts and four readers; Series 4 and Series 5 each have 
just the four texts. A teacher *s manual ($,99) and a test booklet ($.49) 
are included in each series. Supplementary tapes provided for pupils who 
need special help cost approximately $18.00 each. A class record book ($.49) 
allows the teacher to record efficiently pupils^ progress from Book A through 
Series 5. The Behavioral Objectives Manual ($1,49) explains the behavioral 
objectives achieved by the Reading Program, The placement examination cover- 
ing the entire Reading Program Is priced at $.49. 

Programmed Reading (revised edition) , by C, D. Buchanan, is composed of 
a preraading program and three basal reading series. The prereading program 
for kindergarten and grade 1 utilizes alphabet cards, sound-sjrmbol cards, an 
alphabet chart, a preraader, and a teacher’s guide. Most of the material 
is presented by the teacher. Series I, corresponding roughly to grade 1, 
has Programmed Reading Books 1 through 7, 14 accompanying storybooks, a 
student test booklet, and a response booklet. Series II, generally corres- 
ponding to grade 2, has Books 8 through 15 and the same correlated materials 
as Series I. Similarly, Seriaa III has the same sets of materials using 
Books 15 through 24 to correspond roughly to grade 3, This program is a 
basic series, as Is the Sullivan Reading Program described above, but Pro- 
grammed Reading is designed more for general students and leas for inner-city 
pupils than is the other progr^. Webatermastera (dittomasters) provide 
seatwork for each series. 

Comprehension Readers , by Sullivan Associates, contain stories to 
enhance and broaden the pupils’ reading experience at specific levels* They 
also contain questions to develop and test the pupils’ understanding. Four 
Comprehension Readers accompany each text in the Sullivan Reading Program 
Series 1 and 2. Each Comprehension Reader costa $.99, and a complete set 
costs $23*75. 

The Elementary School Reading Laboratory la made up of the components 
of the Sullivan Reading ProgrM, Comprehension Readers, 30 tape reels, and 
the Reading Readiness series. The complete laboratory for 100 pupils is 
priced at $1,995.95, a savings of more than $500 over the price of the 
included materials when purchased separately. 




Project READ , Initiated by Behavioral Research Laboratories in 1968, 
is a systems approach to the teaching of reading; it provides Sullivan 
reading materials, educational consultants, and a parent information and 
Involvement program. With the exception of the McGraw-Hill Programmed 
Reading series, all of the Sullivan reading materials discussed in this 
report are provided as part of Project READ. Teacher training provided 
with the project includes sending BRL consultants to work with Individual 
teachers, and to demonstrate the most effective Implementation of the 
materials. Throughout the school year, the consultants are available to 
work with students and teachers. Project READ costs $20 per student for 

a full year’s program, or $15 per student for a one semester or summer 
session. 




Procedures for Using Product 

Learner Activities 

The uniqueness of the Sullivan materials lies in their programmed for- 
mat and in the linguistically arranged order In which the letters and words 
are presented. Information in programmed texts is presented in small, easily 
mastered steps called "frames.” The difficulty of the material progresses 
very gradually and the order of presentation is carefully logical. In each 
frame, the pupil is asked to supply an answer to a question or to fill in a 
blank. As soon as he has responded, he may uncover the correct answer 
shown in an answer column, which he covers with a "slider" until he is ready 
to check his own response. Reviews are presented frequently and tests are 
provided for systematic .^.urvelllance of the student’s progress. 

Behavioral Research Laboratories, publishers of the programmed materials, 
recommend that other activities be generously interspersed in the periods of 
using the programmed texts. As BRL’s booklet, "Effective Teaching with 
Programined Instruction" (1971) states; 

The experiences of teachers and researchers clearly show 
that an overwhelming majority of students strongly prefer 
progranmed materials to the traditional classroom situa- 
tion. Nevertheless, a steady diet of progranuned instruc- 
tion ±3 not nearly as stimulating for the student as an 

approach that Involved a variety of educational experi- 
ences ip. 5J. 
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To this endj the Sullivan Decoding Kit^ Comprehension Readers , Prograimaed 
Reading storybooks 5 and various tapes have been produced. Other enrichment 
activities planned and reconmiended by BRL but not discusBed ir detail here 
are also available. A reading period might typically include 20 minutes of 
work In a prograimaed textbook, 15 minutes of reading a story, and 15 minutes 
of playing a word game. 

Pupils are not presented with the objectives as they are stated in the 
Behavioral Objectives Manual, but the programed format insures that student a 
approach and master the objectives in the proper order. The teacher Is 
always aware of what sound^aymbol relationships pupils are studying and, 
consequently, can monitor their success in meeting the objectives simply 
by observing their progress through the booklets. 

One of the major characteristics of the programmed materials is that 
they allow the pupil to work Independently and at his own pace. BRL, to 
emphasize the individualization of the materials, urges that "under no 
circumstances should a student be assigned a minimum amount of material to 
be Covered in class" (BRL, 1971, p. 6 ). BRL does recommend, however, that 
minimum standards be set so that no student proceeds so slowly that he makes 
virtually no progress. A slower pupil can be expected to take the prograimned 
books home or to study halls in order to keep up with the rest of the class. 

The possibility of working independently benefits both the slower pupils 
and the brighter ones. In a traditional classroom situation, a slower pupil 
will either interrupt the class to receive extra attention and assistance or, 
because he is embarrassed to admit his difficulty, he will allow the lesson 
to proceed without his undBrstandlng and will fall behind the class. A 
brighter student may find the learning pace of most of the class too slow, 
become bored with the subject, and lose interest. Sullivan reading materials 
then. In allowing each pupil to work at his own speed, can acconmodate both 
of these types of pupil and give them a better opportunity to l^arn To 
relieve the monotony of such a large amount of individual work, many of the 
suppl^entary activities mentioned above provide opportunity for the children 
to work in small groups. 

Teacher Activities 

Because of the programmed format, teachers using Sullivan reading 
materials are given the opportunity to work In the ideal teaching situation. 



a one-tQ^ona relationship with the child. Freed from present informa- 
tion to groups as a traditional reading prograin necessitates^ the teacher 
in a Sullivan class functions as a tutor to each individual child whenever 
that child needs assistance. 

Teacher training is recoimnended as very helpful, but is not absolutely 
necessary. Teachers’ manuals that accompany each set of materials provide 
complete information on using the materials successfully. The advantage of 
teacher training seems to be in the ares of giving teachers guidance and 
encouragement in their often new-found role of tutor. 

The role of the teacher is of utmost importance in a Sullivan program. 
Studies conducted by Sullivan at Hollins College in Roanoke, Virginia, showed 
that pupils had a much greuter degree of success with the materials when a 
teacher provided help and encouragement than when the materials were used 
without a teacher in attendance. BRL (1971) emphasizes this point in their 
teacher Instructions : 

Regardless of the classroom setting, no single factor 
will have a more profound Influence on the student’s 
success than the effect of encouragement and reward 
by his teacher* * # * Make It a point to give as much 
encouragement as you can to every student* * . * Do 
not take the high level of (your students’) perform- 
ance foL granted. * . . The student continues to want 
and need your encouragement — regardless of how well he 
is doing* * * * No program can function with optimum 
effectiveness without drawing on the combined energies 
and resources that derive from the meaningful partner- 
ship of teacher and student [pp* 8-9] , 

Provisions for Parent/Community Involvement 

For most of the materials, no special provisions for parental or 
comamnity involvement are required or included* However, sufficient and 
appropriate information regarding the Individualized and programmed approach 
Is provided so that local districts can inform parents of the goals and 
nature of the program. 

Parental involvement in Project READ Includes a progrOTi to train 
parents as clagsroom aides, to demonstrate how parents can work with their 
children at home, and to encourage parents to visit the school to discuss 
their child’s reading progress with the teacher, 
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special Phygical Facilities or Equipment 

No additional equipment other than that supplied by the product is 
required, nor any special facilities beyond those typically in the class- 
room. 

Recoaimended Assessment Techniques for Use rs 

Specific behavioral measures of intended achievement are provided in 
the materials. 



ORIGINS 



Key Personnel 

Although the Sullivan Reading Program development relied on the contri- 
butions of hundreds of people, three individuals stand out as the major 
developers. They are Dr. M. W. Sullivan, Dr. Allen Calvin, and Miss Cynthia 
D = BuchsuBn » 

Dr. M. W. Sullivan was the developer of the Sullivan Reading Program 
and is currently a member of the Board of Sullivan Associates and Director 
of Research for Behavioral Research Laboratories. 

Having served in the U.S. Marine Corps from 1943 to 1946, Dr, Sullivan 
returned to an interrupted college career to earn a B.A. and M.A. in English 
from Yale University, a B.A. in Spanish from the University of Puerto Rico, 
an M.A, in Spanish from Middlebury College in Vermont, and a Ph.D, in 
linguistics from the University of Madrid. His academic experience includes 
Master of Modern Languages at the Choate School in Wallingford, Connecticut; 
Instructor in German and English at the University of Puerto Rico; Lecturer 
in Linguistics at the University of Madrid; Instructor in Spanish at Yale 
Univeraityi and Assistant Professor of Modern Languages at Marquette Univer- 
sity in Milwaukee, Wisconsin. In 1954, Dr. Sullivan joined the staff of 
Hollins College where he held the positions of Associate Professor of Modern 
Languages, Director of the Language Laboratories, Professor of Modern 
Languages, Head of the Modern Language Department, Director of Graduate 
Studies, and Director of Foreign Language InstltuteB under the National 
Defense Education Act, 

Dr. Allen Calvin is Chairman of the Board and Chief Executive Officer 
of Behavioral Research Laboratories, as well as Director of Research for 
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Sullivan Associates. He received his B.A. in psychology from the University 
of Minnesota, his M.A. in psychology and his Ph.D. in experimental psychology 
from the University of Texas, Dr. Calvin has been a Research Scientist with 
the U. S. Air Force and an instructor in psychology at Michigan State 
University. In 1956, he joined the staff of Hollins College, where he held 
the positions of Assistant Professor and Acting Chairman. Articles published 
before the beginning of the Sullivan Reading Program Included s "The Growth 
of Learning During Non-Differential Reinforcement," "Perceptual Differentia- 
tion in the Course of Non-Differential Reinforcement," "The Effect of Delay 
on Simultaneous and Successive Discrimination in Children," "Configurational 
Learning in Children," "The Relative Efficacy of Various Types of Stimulus- 
Objects in Discriminative Learning by Children," "The Effect on Non-Dif feren- 
txal Reward and Non-Reward on Discriminative Learning In Children," and 
"Spoken and Written Vocabulary; Their Relation to Standard Vocabulary Test, 
Intelligence, and Anxiety," Complete referencea to these articles are 
provided in Appendix C, 

Miss Cynthia D. Buchanan is President of Sullivan Associates, having 
succeeded Dr. Sullivan In that position in 1970. As a freshman at Hollins 
College in 1955, Miss Buchanan was Dr. Sullivan's advisee. He recognized 
her ability and enrolled her in advanced linguistics courses. She received 
her B.A. in French from Hollins College, and her M.A. in linguistics from 

Harvard-Radcliffe. She returned to Hollins in 1959 as an assistant instructor 
in modern languages. 



Sources and Evolution of Ideas for Pr oduct 

The origins of the Sullivan reading materials ^tended back to World 
War II, when Dr. Sullivan was serving in the Marine Corps. At the beginning 
of the war, the United States discovered that they had few people well 
enough trained in foreigri languages to do intelligence work in Europe. Rare 
was the American who could speak French well enough to convince a German that 
he was French, and rarer still was the American tdiose German could fool a 
German. American intelligence agents dropped behind enemy lines were 
recognized as American almost immediately because of their lack of foreign 
language accomplishment. To rectify this embarrassing situation, linguists, 
a not very common group themselves, were drafted to ^et up methods of teaching 
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foreign languages rapidly. The courses they developed initially were 
repetitive and dull, according to Sullivan, but they eventually evolved to 
become the Holt-Heath courses in current use. 

Through these efforts of the Armed Forces, Sullivan, at that time having 
done two years of undergraduate study in linguistics in one year’s time, 
became involved in teaching German to Americans. One of his students* tasks 
was to convince German scientists to come to America rather than go to Prussia 
at the end of the war. 

German scientists at that time were technologically more advanced than 
were Americans. One of the products of German technology was the tape 
recorder, a device that had not yet been introduced In America. Sullivan 
recognized the great potential for the tape recorder in the teaching of 
languages, and he put them to use with his language students. Sullivan 
asserts that he was the first person to teach using the machine. 

At this early point in his career, Sullivan discovered that standard 
lecturing and drilling techniques were not very effective in the teaching 
of language, so he switched to a technique utilizing a dialogue between 
teacher and student in order to discover the student’s learning problems 
and respond to than. He did not, however, yet realize clearly the principles 
that; (1) learning comes from student response; (2) eKact and immediate 
feedback is necessary for efficient learning; and (3) students need to do 
wall from the beginning or they will became discouraged and lose interest. 
These are three major principles of learning theory. 

Upon returning to civilian life in 1946, Sullivan ntinued his studies 
in linguistics at Yale and set up the first "language laboratories" ever 
developed. He used Webcore wire recorders in laboratories that he built 
with his own hands, drawing upon his experience with radar in the Marines. 

The laboratories he set up consisted of a series of listening posts plugged 
into one machine that played a tape he had recorded. This work continued 
during his years in Puerto Rico and Madrid (1949 to 1952) , but he felt that 
the language laboratories were a failure since no one seemed to be learning 
much from them. Upon returning to the United States, having given up hope 
of making language laboratories work, he dlacovered that the idea was just 
gaining in popularity here, Sullivan states that It was not until about 
1954 that other language laboratory advocates decided the method was 
unproductive. 
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During the years that Sullivan was striving to make language laboratories 
work, the field of psychology, and more specifically, the area of learning 
theory, were progressing Independently and without much attention from him. 

A variety of past research paved the way for Sullivan's future work. Behav- 
ioristic learning theory came into being in approximataly 1913, when John B. 
Watson pioneered in the field of behaviorism. In the 1920' s, S. A. Pressey 
developed something like a teaching machine while trying to make a testing 
machine. In the 1930' s, psychology really became a predictive science for 
the first time with the publication of Clark L, Hull's formulas on habit 
strength. His formulas, based on schedules of reinforcement, could predict 
specific behaviors, B, P, Skinner, too, proposed a system of behavior, but 
it varied considerably from conventional stimulus-response psychology. His 
theories were not well accepted until the 1950' s. 

Much, then, was known, about how people learn and a great deal of research 
had been done, but until the 1950^ s, few had applied what is known about 
Isarning to the classrooin, the place where learning supposedly occurs, 

Sullivan relates the following anecdote showing the transition from 
learning theory in principle to learning theory in practice J 

In 1954, B. F. Skinner visited his daughter's elementary classroom, 
perhaps becoming the first learning theorist to discover the difference 
between what psychologists believed to be conditions favorable to learning 
and the conditions of a supposed learning situation. Skinner was reportedly 
outraged when he saw that the school's most lauded teacher taught in a 
manner that would make children hate school and would hinder learning more 
than help it. Confident that he could do a better teaching job, he went 
home and set up the first "program." His philosophy was to present a 
stimulus, allow the child to respond, give him immediate feedback to show 
his response as correct or incorrect, then use what the child had learned 
to generate another reaponae. Naively, Skinner believed he could simply 
sit down and write an adequate program. 

In Skinner's brush with reality, he had discovered that from a learning 
theorist's viewpoint, education had remained virtually unchanged in centuries. 
Still in use was the lecture system, which Sullivan describes as "someone 
standing up and talking to himself." Sullivan cites a 1960's series of 
studies that showed that only 20% of an audience is listening to a lecturer 
at any given moment, and that is under ideal listening conditions. Students 



are given no opportunity to respond in the lecture situation and if, as 
learning theory states, all learning is in the response , certainly very 
little learning is occurring. 

When Allen Calvin Joined the staff of the Hollins College Psychology 
Department in 1955, he was already very widely publ._ahed and recognized as 
a leader In psychology. He and Sullivan came to be close friends, and 
together, they published articles and did studies on the phenomenon of 
anxiety. Through Calvin, Sullivan learned of B. F, Skinner's work with 
learning theory and prograiming * 

Sullivan reports that until he became aware of Skinner's work through 
Calvin, he, Sullivan, had been dissipating his creative energy in the writing 
of plays, poems, articles, and in acting and directing. He was currently 
Associate Professor of Modern Languages and Director of the Language Labora- 
tories at Hollins College, but his career had no center, no focus. When he 
learned of Skinner's work, he was fascinated and very quickly became addicted 
to programming." The friendship and professional association between Calvin 
and Sullivan resulted In substantial contributions to the field of psycho- 
llngulstics . 

Sullivan had long been an advocate of learning without the intervention 
of teachers. He thought that materials should teach, not people. With pro- 
gramming as a new source of inspiration and Calvin encouraging him to develop 
the technique as a teaching tool, Sullivan went back to his earlier language 
laboratories and did the lessons all over, using the programmed format. It 
was in the course of developing foreign language programs that Sullivan was 
motivated to produce prograiomed reading materials. 

Sullivan tested his new programs on local high school students. He 
found, however, that the students made large numbers of errors in his programs, 
and the programs were designed to elicit nearly all correct answers. He was 
sure that his programs were not at fault- the high school students simply 
could not read well enough to follow the programs. The fact that students at 
that level could not read adequately infuriated Sullivan, and ha was deter- 
mined to remedy the situation. He began his work in the field of teaching 
reading. 

Sullivan used standard reading programs and set them up in a progranmied 
format using teaching machines (specially adapted tape recorders), but even 
the brightest children who tried thai couldn't learn much. When standard 
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reading programs, set up according to the principles of learning theory 
(stimulus-response-reward) to maximize their potential, could not teach 
children to read, Sullivan concluded that pupils using the reading programs 
in their normal Icrinat were learning virtually nothing from them; all learn- 
ing was coming from the teacher. Sullivan then set up selected Superman 
comics in the programmed format and using the teaching machines, and he 
found that children learned reading much better from the comics than from 
the standard readers! 

Studying the standard reading programs, Sullivan discovered that they 
made absolutely no use o:f the principles of linguistics. Linguistics 
breaks down language into "phonemes, " the individual sounds used to form 
words. After the phonemes have been isolated, a "grapheme" is assigned to 
each phoneme. A grapheme Is the symbol that represents the sound of the 
phoneme. Linguists, having studied the English language, know which sound- 
symbol combinations are the most comnion in our language, and which are the 
most unusual and, therefore, most difficult. The authors of standard 
reading programs, however, seemed oblivious to this area of linguistics, 
and the vocabulary presented was a hodgepodge starting with the wildest 
and most irregular words in the language. The practice of authors had been 
simply to accept, the vocabulary list set up 150 years ago by McGuffy, "who 
undoubtedly got it from God," says Sullivan. 

The common practice has been to teach common words, even if they were 
one-of-a-kind in the language. Whan a child learns "does," he learns that 
"oe" has the sound of "uh." But when he comes to "shoes," or any other word 
using "oe," and pronounces it "uh," he will be wrong; teaching such words 
as "sight" words leads to 100% negative transfer. The child cannot extend 
his knowledge to master new words, Sullivan describes this method of learn- 
ing sight words by visual perception as being the same as saying, "I can fly 
if I simply flap my arms hard enough to take off." 



Funding for Product Development 



Funds for Sullivan* s programiiiing efforts came from a variety of sources. 
The U. S. Office of Education, General Motors, and U. S. Steel all provided 
grants to Individual people to do independent research. The NDEA Foreign 
Language Institutes Sullivan directed in the summers of 1959 and 1960 pro- 
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provided hund’"eds of thousands of dollars and a fleet of tape recorders and 
t?n©s. Sullivan regards a large grant from the Carnegie Foundation in 1959 
as the "seed" grant that actually produced the Sullivan Reading Program, 
although a great deal of research work in programming had been done before 
that grant was received. 

The Carnegie grant, "a substantial sum," to quote Calvin, was originally 
Intended to be used to improve the teaching of foreign languages in the class- 
rooms with programaed materials. Shortly thereafter, however, the grant was 
expanded to include reading and mathematics programs. A massive, later grant 
from the Encyclopedia Brittanica Films allowed them to build new facilities 
and to expand progr amm ing efforts into the areas of science and geography. 

Also in 1959, Hollins College was becoming more interested in Calvin’s 
and Sullivan's work. Pleased to have its two most highly regarded professors 
doing research that was being acclaimed, Hollins also gave them funds. This 
grant made it possible to hire Miss Buchanan as an assistant Instructor in 
Sullivan's department. Calvin estimates that a total of $3.5 - 4 million 
was spent in research and development, perhaps one-third of which went 
directly toward the reading materials. 



PRODUCT DEVELOPMENT 
Original Development Plan 

Sullivan, Calvin, their associates, and assistants were simultaneously 
developing so many programs in such a variety of subject areas that it is 
impossible to isolate the development of the reading roaterlals from the 
others. Of the thousands of people Involved in the development and evalua- 
tion of the materials, certainly not all were direct] y connected with the 
reading progr^, but research done in all subject areas was utilized in the 
development of the reading materials, 

Calvin was responsible for setting up the programming patterns and the 
complicated flow of production. His was also the task of restructuring the 
school world to acconmodate the new materials that would be produced. This 
latter responsibility was an important one. During the early stages of 
development, programmed materials were viewed with alarm and suspicion. 

They were seen as a threat to the existing educational system where the 
teacher was the focal point of the class and the provider of basic information. 



With prograinmed materials j it was feared the teacher would become obsolete j 
unneeded when progranmied books could provide the basic information, Calvin 
also contributed suggestions about how people learn ^ and gave seminars in 
psycholQgy * 

From the very beginnixig^ Calvin realized that hundreds of staff members 
and a complex routing and rerouting system for programs would be needed. 
Until 1959s Sullivan and Calvin had difficulty finding someone to direct 
the complex system. Then, however, they hired Lewis Miller as production 
manager and he succeeded in keeping the complex procedure operating. 

Sullivan was to be the actual programmer of the materials and director 
of the programming assistants. His role was much more that of a creator and 
writer than that of an administrator, Sullivan had the dogged per severance 
that was required to produce frame after frame of programmed material ^ only 
to have it criticized as "trash" and to have himself called "crazy." 

Miss Buchanan had a USOE grant to do research in automated teaching of 
descriptive linguistics. Having just received her master's degree, she says 
she was "full of the principles of descriptive linguistics" and was very 
enthusiastic about working on reading programs since no reading materials 
on the market at that time involved these principles. She waSp consequently , 
to do an independent research project in reading. 

One of the major goals of the programmed materials was to keep the 
child Interested in what he was doing. To this end, Calvin’s production 
flow plan Included extensive formative evaluation periods. Each program 
would be triad out by youngsters. Their error rates and comments on each 
frame of each progr® would be recorded , and the program would be revised 
accordingly. Plans called for each program to be tested and retested with 
additional pupils until no further Improvement in error rate or interest 
level could be mada, 

Under the Carnegie grant and during previous years of independent 
research, Sullivan and his progranming staff were not looking toward publica^ 
tion. They were interested solely In research and in developing successful 
teaching techniqueSi When they started, they were young and idealistic and 
concarned with doing "something beautiful, wonderful, and good," They had 
lots of ideas about things they wanted to do in reading, but no time schedule 
and no definite plan of development. In the last year of the research 
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project, Sullivan perhaps had some idea of future financial gain, but 
according to Miss Buchanan, most of the staff was simply enthusiastic about 
the research for its own sake. What has evolved is a multi-million dollar 
eonunercial venture that alms simultaneously to improve children’s learning 
in as many subject areas as they can find time and people to work in and to 
make money. 

In the beginning, they really didn't know what they would wind up with, 
except programmed materials. Sullivan and his associates still do not know 
what they will wind up with for the work continues at a hectic pace and 
Sullivan progrananed materials are spreading to all kinds of schools all 
over the country. Their work began as empirical research, and only later 
did they envision the variety of products they now produce. 



Actual development can be said to have begun in 1956 with Sullivan's 
first efforts at producing programmed courses in modern languages. Sullivan 
and his staff generally worked In several areas besides reading, but the 
research that eventually evolved into reading materials was done and the 
techniques were refined in all of the subject areas. 

From 1956 until 1959, Sullivan developed his programming techniques 
mostly in the areas of modern languages. He found that his earliest pro- 
grams, even in their crudest form, proved to be far more successful than 
Sullivan himself operating as a regular classroom teacher. Even the best 
teachers operate under severe handicaps within the standard classroom altua- 
tlon. The only feedback a teacher usually gets, in Sullivan's terms, is "supersti- 
tious behavior." The only imiediate response to a teacher's lecture comes 
in the form of smiles, nods, or questions from the class. If a student 
smiles at a particular point in the lecture, the teacher is reinforced | 
he remembers that point, perhaps emphasiEes it, and uses it again the next 
year. That point may have been terrible and the smiling student may have 
bam thinking of last night's date, but that lecture point is there to stay, 
regardless of its merits. Similarly, the teacher's entire teaching strategy 
is made up of a series of such superstitious behaviors, Sullivan, as a result 

*See Figure 1 for a diagram of the major events in the history of the product. 



Actual Procedures for Development of Product^ 
and Performance Measures, 1956-1961 
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Figure 1 
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of hla early tests, knew his programmed lessons could teach better than even 
the best of the regular classroom teachers . 



In the early months of programming, Sullivan and Calvin initiated 
experiments to determine what types of pictures, characters, and subjects 
appealed most to children. The results indicated that the pupils liked 
simple, cartoon-type pictures with direct referents for each noun and 
operations for each verb. They wanted each picture to be linked closely 
with a simple sentence so that the reading matter was meaningful to them. 
This information led to direct revision of the materials. 

The years of 1957 and 1958 represented one of the most important phases 
of prograimning research. It was then that Sullivan and his associates found 
the best working format to use. All of the early prograns relied on tape 
recorders. The programs were mimeographed sheets that had been colored in 
by hand. The sheets snapped into a notebook, and a slider to cover the 
answer colunm was used. Each time the child came to a star on the page, he 
would press a button on the tape recorder and listen to oral Instructions. 
Much of the early materials were tremendously repetitive. The three frames 

from one of the first experimental mathematics programs below are an 
example . 



6 


The set of numbers from 5 to 7 consists of the nimbers 
5, 7. 


7 


The set of numbers from 5 to 7 consists of the numbers 
5. 6, 


5 


The set of numbers from 5 to 7 consists of , 6, 7. 



This format followed the Skinnerian program block, wherein each frame was a 
totally independent unit. Each frame, therefore, had to include all of the 
Information necessary, even though the previous fr^e had Included it. 

Results indicated that the children becMe bored with such repetitive 
sequences, and one of the major decisions of the staff v;as to revise the 
format. In 1958, Sullivan invented "chaining," the building of each frame 
upon the previous one. Information could be presented just once instead of 
over and over. A program to teach people how to play chess was the first one 
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actually to use the new "chained" format, but the reading materials quickly 
adopted It. 

The early stages of progr ammi ng In 1956 and 1957 made use of an early 
manufacturer's entire production of teaching machines, and Sullivan's staff 
produced their own as well. Neil Sullivan, Sullivan's brother, had come to 
the field of psychology from engineering, and he developed machines especially 
for the programmed materials. One machine used a film strip with rear pro- 
jection onto two screens placed in front of the child. A tape cartridge 
would trigger the filmstrip changes with subsonic impulses. 

All of the machines were expensive, so not many purchasers could buy 
them, and easily broken, so much of the time they were inoperable. Each 
time the progrMi was revised, a new filmstrip and a new tape had to be pro- 
duced, Sullivan and his colleagues finally realized that the machines "only 
enriched the people who made machines," In Sullivan's words, and all of the 
materials were revised to function without them. 

A third major format change Involved the organization of frames into 
programs. The original format divided each page into several horizontal 
sections. 
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Instead of reading down the page, the child read line 1 and answered It, then 
turned the page to find the answer, then read line 1 on the next page. Thus, 
the child read entirely through the book using only the first row of each 
page. Then he would start over and proceed similarly with the second row 
on each page throughout the book. Sullivan felt this format was unsatis- 
factory, He proposed that the stimulus and response columns be placed on 
the same page and that the child use a cardboard slider to cover the answer 
until he had responded to the question. To see if the proposed format would 
actually be an improvement, Sullivan ran a two-week study of matched groups, 
one group using each format. 
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:^ru« At i f fehe old format worked' better* : Sullivan believes that the 
children liked the opportunity to turn the pages so quickly, and to get' 
Lthrough the,- book, ao .fast. ..He supposes ;that they really felt, they Were 
accomplishing something* . By .the and of the two-weeks, however i the children 
tired rof fpandlng so .much l time turning pages; ih psychologists ^ terms, ‘ a 
reactive inhibition buildup had occurred:. :.Also problematic with! the old 
•7 was .that the time between th8;:child^s :respQnse and ’hls"'reinforceient 

Cseeing the correct response on the .neKt page) ; was: too- long. he study 
resulted in the support of Sullivan’s new. format* Post tests. ho wed ^ that 
.children who had used it had learned better than. thoBe who > had usM the old 
format, , • ; yririd -l . 7:> ' ■ V; . 

r- : . As Head of the = .Modern language Department and. Director v of vGraduata ^ 

.Btudies at Hollins., Sullivan put his entire staff to work on^ programming 
The faculty^ graduate students, and even some undergraduates 
ware assigned to do programmingvresearch*-Galvin’sentirepsychologydepart- 
j^was^involved a/Jj by* v*:! ‘ •■ii i 

XiiinoliR i^with thfi jg^rftnt iiigi of '.the’- Carnegie funds, the ^pf ogratiiming^Opera— 

tion underwent an Immense expanaion. Scores of personnel from outside the 
college were enlisted, Carnegie funds allowed Sullivan to hire the beat 
professors in the areas he^ished’^^o- program. team of linguists, a team 
of prograimiers to assist EuiUvan with thej actual writing, a group of psychol- 
oglsts, a series of subjec#HP^tte^-^ecialtBtB“,j and dozens of teachers in 
local schools were assemb]|e#"tO’^rotiuce^"an^^esjl: programs. Their objectives 
were quite specific: They- Were -to producet^ogramme^ materials for languages. 



mathematics, and reading ani-‘te-^r4ng-‘-them;^out"'of the laboratory and into the 
classroom. 

bs^evanB bna J snxi besi i^rU mmh ^nlbBo’X to 

,au::s j° have ^y^^ploglsts 

determ^ 

,Jrequ.#ntly able 

to add Insights into reading behavior from their comion experiences. 

-‘7n7 yrr;# 7- . ^ / •. y. ■ ■. _ \ - ;■ 

, .... had been _adapted enough so that everybody agreed on it, 

. fte^ o programmers would write the flpst version 

of the program. The program then went back to the other groups , who would 
"comment, suggest, and curse," as Sullivan reveals, then be returned to the 




progranmlng team. This cycle was repeated at least six times before a child 
ever saw it. At this point, says Sullivan, the program was still ''trash,” 
but at least it was something the child could try out. 

Sullivan's initial method of programming was to write or type the pro- 
grams himself. Giving tliat up as too laborious, he tried dictating the pro- 
grams to a superhuman typist. Cognizant of the fact that the typist's time 
was not being well spent, Sullivan at last accepted Calvin's suggestion that 
the programs be dictated to a machine, then taken to an assistant to be 
written up. 

From 1S39 to 1961. Sullivan and Miss Buchanan were both ensased nrimarily 
with research for reading materials. Miss Buchanan was working without 
salary from the Carnegie grant, although her supplies and office space were 
covered by it. Her research was toward reading materials to teach beginning 
pupils to read. She describes this time as ''exposure to the field,” and 
as a period of refining programming techniques. Working with children in 
the reading laboratory at Hollins, she discovered that even before she could 
think about teaching them reading, she would have to teach them colors, 
numbers, and directions. Sullivan acted as a reviewer, director, and editor 
of Miss Buchanan's work there. 

Simultaneously, Sullivan was doing preliminary work on a reading program 
of his ovni. His materials utilized a different progression of sounds than 
did Miss Buchanan's, and his work was designed more as a reinedial program 
for children who had tried but failed to learn to read. Miss Buchanan 
relates that, "He worked like a madman while he was at Hollins,” writing, 
testing and rewriting day and night. "His capacity for work was Just 
incredible. During these years, Sullivan was also writing programs in other 
areas as well as directing the hundreds of assistants on his staff. 

In the early days of programming development, Sullivan and his colleagues 
who were so dedicated to progrmiUEilng were looked upon as fanatics. Prograimning 
sefflQed to have no comii»rclal significance. After the Carnegie grant pushed 
programming more into the classroom arena, however, it became probable that 
prograramlng might be a worthwhile commercial venture after all. Encyclopedia 
Brltannica Films saw the potential in programmed materials and proffered 
millions of dollars. Using that and the Carnegie money, Sullivan built a 
new facility at Hollins College In 1960 and employed more than 700 people. 
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By this time, Sullivan and his programmed learning staff were by far 
the largest and most Important project at Hollins College, which had less 
than 1000 students of its own. Every year, Sullivan received more grant 
money for his own projects than the rest of the college had received in its 
whole history. Sullivan's assistant programmers made more money than full 
professors outside his group did. Programming proved to be very threatening 
to the regular professors, as well. It was known that by working with pro- 
grammed materials at Sullivan's psycholinguistics laboratory, subjects could 
be learned more easily and more thoroughly than in a professor's class. 

Under stanclably, relations between Hollins College and Sullivan's staff 
were strained. The Encyclopedia Britannica Films money was given to Sullivan 
through the dean's office. The dean did not understand Sullivan and his 
programming madness and Interfered with his operation. 

Sullivan, Calvin, and Encyclopedia Britannica Films all were ready for 
a change, and Encyclopedia Britannica offered to build a new canter for the 
programming activities somewhere else. Sullivan and Calvin felt that a new 
arrangement with EBF away from the college would place fewer restrictions on 
them, so the prograraned learning capital moved from Roanoke, Virginia, to 
Palo Alto, California. 

Calvin selected the new site for several reasons. He was familiar with 
the San Francisco Bay Area and liked it as a place to live and work. Further, 
the Industrial Park in Palo Alto was closely associated with Stanford 
University and provided a particularly good arrangement for organizations to 
share in the acad^ic coiuiunity. Consequently, Encyclopedia Britannica Films 
built a new facility, The Britannica Center for Studies in Learning and 
Motivation, in the summer of 1961. 

There seams to have been little disappointment at Hollins College that 
Sullivan, Galvin, and their asBoclates were leaving and taking the programmed 
materials with them, Hollins, a small, exclusive women's college, wished to 
avoid all the publicity that the progrmnmed materials were attracting. 

Neither did the college wish to expand to acconmiodate Sullivan's personal 
needs . 

Calvin was Director of the Palo Alto Center , and Sullivan was Associate 
®^tector, Sullivan s faculty from Hollins and some of his students were also 
transferred. Unfortunately, the new arrangement was less than had been hoped 
for. Encyclopedia Britannica Films sent their own business managers to run 



things, but the business staff and the research staff were unable tc work 
together or understand each other. Sullivan’s staff was academically 
oriented, the corporation was comnerclally oriented. Having opened in 
September, 1961, by December Sullivan’s staff members had either resigned 
or been fired, and the Center closed. In hindsight, Sullivan states that 
It was the "best thing in the world to have happened." The parting of 
Sullivan and EBF was not, however, quite as agreeable as the separation 
from Hollins, Sullivan sued EBF for breach of contract. A rather messy 
legal battle seems to have followed, the details of which are not known. 
Sullivan seems, however, to have had some degree of success, for his staff 
received a year’s salary. 

Formative Evaluation 

Thousands and thousands of Roanoke children from the city and county 
schools were used to test the materials. Children of the Hollins faculty 
were used the first time through a program, Sullivan describes them as 
"kids who could get everything right," The first tryout required children 
who could make enough progress through the program to produce an error rate 
On each frame and a collection of comments. 

After revision by Sullivan and the programming team, the program was 
tried out on bright kindergarten pupils from the Roanoke public schools. 

This second tryout would hopefully result in more favorable comments and 
fewer errors. The third and fourth tryout phases for eacu program utilised 
rural children and black children. "They were the most disadvantaged kids 
anywhere," says Sullivan. Their parents were generally illiterate and their 
homes were huts without running water. 

Each version of the program was different. Frames were added and expanded, 
and pictures would change. Programs generally grew in length during the 
revision stages. Children trying out the materials were observed through a 
one-way glass to see how they proceeded. The early stages also represented 
a period of deeiding what vocabulary the children could master most easily, 
and what words had the most meaning for them. 

The first version of each program generally consisted of what the staff 
thought and felt. By the time the program had undergone months or years of 
testing and rewriting, a tremendous amount of pupil input and data had been 
incorporated. Development of each program represented continuous progress 
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from its somewhat haphazard beginning to the time when it was accepted as 
"good.” A good program was one that resulted in 19 of 20 frames eliciting 
correct responses. 

Not all of the programs developed turned out to be good. Often, after 
months or years of work, a program was discarded If it did not come up to 
Sullivan's standards of acceptability. In this respect, the prograumed 
materials had a great advantage over other textbooks that seldom if ever 
undergo developmental testing. By the time Sullivan's materials reach the 
public, they have been proven to be effective by peers of the people who 
will use them. 



Actual Procedures for Development of Product 
and Performance Measures, 1961-1970 

Development 

In December, 1961, Calvin and Sullivan set up their own operation in 
the form of two separate companies. Calvin organized Behavioral Research 
Laboratories in Menlo Park, California, while Sullivan founded Sullivan 
Associates in the hills behind Palo Alto. Behavioral Research Laboratories 
(BRL) was initially funded by $40,000 in private Investments. Organized 
to develop more programmed materials in all areas, BRL also has evolved into 
the publisher and marketer of nearly all of the Sullivan progranmed materials. 

The founding of Sullivan Associates by Dr, Sullivan represented a "do 
or die" situation. Sullivan used about $60,000 of his personal funds to 
start the company; he was "putting his life on the line, both professionally 
and monetarily," according to Lewis Miller, Sullivan and Calvin were both 
so certain that progranmed materials worked and would be well accepted by 
the public that they had few doubts about plunging into the world of business 
with their products and ideas. Publishers, including McGraw-Hill and 
McMillan, made monetary advances to Sullivan, too, on the basis of his repu- 
tation as B. I. Skinner's brightest protege. 

The two corporations are "so interrelated, its terribly incestuous," 
according to Calvin. Though separate, they are totally interlocking. BRL 
was set up with Calvin, Sullivan, Miss Buchanan, and John B, Hanriksen as 
the Board of Directors. Calvin, Sullivan, and Henriksen made up the Board 
of Sullivan Associates, as well. Sullivan Associates has been almost entirely 
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responsible for the actual development of the reading materials, however, 
so BRL will receive little further consideration in this report. 

ApproxlDiately 20 people made up Sullivan Associates permanent staff 
in the beginning. Sullivan was President, Lewis Miller was Executive Vice 
President, and Cynthia D. Buchanan was Secretary-Treasurer. The nucleus of 
Sullivan's programming team at Hollins made up part of the staff. The 
remaining staff members were all recruited from among Stanford University 
students and graduates. Some held Ph.D.'s; many aecretaries had earned 
master's degrees. 

Staff relationships under Dr. Sullivan are relevant to the development 
and success of the programmed materials. He does not seem to have been an 
easy man to work for. Brilliant and completely devoted to his work, Sullivan 
had high expectations for the quality and quantity of work produced by his 
staff. Reportedly impatient with people who could not think and snobbish 
about academic degrees of his own and of his staff , Sullivan demanded per- 
fection from them. He speaks with disgust of professors who came both to 
Hollins and to Sullivan Associates unwilling or unable to invest the time 
and energy required to write programs. He had no trouble finding people to 
hire; working for him was an honor and a challenge, and If an assistant did 
well, he would find Sullivan charming and exciting to work with. If, however 
an assistant did not do well, there could be no working relationship and the 
assistant would have to leave. Little personnel turnover occurred during 
projects, according to Lewis Miller. Staff members came and went, but mostly 
between projects and because of their own interests. 

Each project at Sullivan Associates had its own staff, but all staff 
members contributed to and criticized each other's work in regular "critique 
sessions." Subject matter specialists, hired as subcontractors for specific 
projects, had their own specific areas of responsibility, but no one worked 
exclusively on one thing. Since its founding, approximately 90^ of the work 
at Sullivan Associates has been toward development and refinement of reading 
materials. 

With the founding of Sullivan Associates, M. W. Sullivan removed himself 
from the cloistered life of an academic researcher and entered the world of 
business. Sullivan and Calvin and their assistants came to the realization 
that they were experts in the field of learning, and decided It was time to 
advance progranmiing from the research and laboratory stage and put it into 



operation in classrooms ■ Many men would have had great difficulty in making 
the adjustment from being a professor and researcher tc becoming an adminis-“ 
trator, but Sullivan met the problems with, aplomb. He examined each new 
task carefully, dissected it, then studied the subject at hand systematically 
until he had mastered it, Lewis Miller offers as an example of Sullivan^ s 
ability to cope with the business world the story of Sullivan’s meeting with 
a publisher i At their first meeting, Sullivan was inexperienced in the field 
of publishing. He listened carefully to all that was said but said little 
himself. Before the next meeting with the publishers, Sullivan read and 
studied about publishing. At the second meeting, he knew more about the 
field than did the publishers and asked questions they could not answer. 

Then he took the contract they had offered, rewrote it to suit himself, and 
had them accept it. 

The function of Sullivan Associates was, like that of BRL, to develop 
programmed materials in all fields. In 1961, much of their course had been 
predetermined by the research that had taken place at Hollins. Sullivan 
and Miss Buchanan had both completed most of the research necessary for the 
two different reading programs . They had brought with them the research 
notes and pupil booklets they had developed at Hollins. Their programs had 
been worked on for years and had been tested and retested. All that remained 
to be done was to find the most attractive way to present the programs and 
to do a polished rewrite of the programs that they knew worked. At Sullivan 
Associates, Miss Buchanan wrote from her two years of Hollins research the 
complete series of 24 books that comprised the basic part of the first 
Programmed Reading series. As he had in the past, Sullivan acted as editor 
and critic of this field test edition. 

As the writer. Miss Buchanan oversaw production activities for her 
materials too. Early production procedures were Informal; she directed and 
’■galley slaves” did the illustrating and pasting up. Having the production 
phase so closely related to the development phase was a mixed blessing, 
according to Miss Buchanan. She did not know much about production, a© 
mistakes Biid problems were inevitable* On the other hand, the developers 
of the materials were able to develop all their own formats and designs so 
that the form and content would mesh by using such a production system. 

While Programned Reading was undergoing its firial writing and field 
testing in 1961-1963, as described below, Sullivan did only occasional work 



on his own reading program. He did, however, work on techniques and applica- 
tions of the materials for remedial pupils. Using some of the pupils at the 
Programmed Reading field test schools who had reading difficulties and were 
reading far below their grade levels, Sullivan tested various page formats 
using folders of loose-leaf materials. During these trials, Sullivan decided 
on a new box format that was later used in many other programmed materials. 

When he was satisfied that he had found the best format and that remedial 
reading pupils actually found success and enjoyment with his materials, he 
wrote the final version of the Sullivan Reading Program and delivered the 
draft to Behavioral Research Laboratories for production. Calvin assembled 
some thirty artists to prepare the program for publication. The actual 
publication and distribution of the two reading programs will be covered In 
the Diffusion and Adoption sections, below. 

Decisions about what supplementary materials to develop were generally 
up to Sullivan. He did not use performance measures to determine if the 
children needed a certain type of enrichment or supplementary material. 

Neither did he ask knowledgeable” people if they thought the materials were 
needed. He trusted his own Intelligence, experience, and intuition. As 
Miss Buchanan points out, however, Sullivan is a rather extraordinary man. 

When you are the expert, you do not need to ask the opinion of people who do 
not know nearly as much about your subject as you do. 

As they began to supplement the basic reading programs, Sullivan carried 
both the administrative and creative responsibilities of Sullivan Associates. 
Miss Buchanan carried much of the creative burden, too; however, Sullivan 
would give the original directions and ask Miss Buchanan to develop a plan 
for the project. Together, they refined the plan. She was then responsible 
for executing it, either as sole author or as principal author with some 
assistance. Sullivan then would give final approval and the product would 
be field tested. Each part of the materials is generally the result of one 
person's work, with lots of help from others. 

Prepared In this manner were the following programs J Sullivan Story- 
books Cto accompany Programmed Reading) whose publication began in 1965 and 
continued until 1971; Sullivan Decoding Kit; Readiness in Language Arts 
Program, the Enrichment Materials, and Reading Readiness, all of which were 
published in 1967; the I Can Read series, published in 1968; and the Com- 
prehension Readers, published In 1970. Several of these projects were always 
in progress. 



In 1966, Sullivan Associates submitted Programmed Reading (distributed 
by McGraw-Hill) to the California State Adoption Committee. The program was 
not accepted for adoption, so Sullivan Associates spent 18 months revising 
the program. Other considerations were also involved in the decision to 
rewrite the materials. The Cleveland schools had expressed a desire for a 
better racial balance within the series. Sullivan Associates had learned, 
also, since the first publication, that learning was improved directly as 
the number of pupil responses were increased. Consequently, Progratmned 
Reading, the only basic reading series to be revised before it had been on 
the market five years, was revised, starting in 1966. Black, brown, and 
oriental characters were added to the series and 40% more responses were 
added. Diagnosed tests were interspersed at regular intervals to aid teachers 
in grading and diagnosing individual progress. The prereading materials 
were also revised, to accommodate the trend toward teaching reading in kinder- 
garten, and the prereading program was simplified and streamlined to intro- 
duce pupils to words and simple sentences earlier. Further, three years of 
use had demonstrated that the teachers needed more detailed instructions, 
so the teacher’s guides were rewritten. 

From the mid-1950's until 1970, M. W. Sullivan had devoted virtually 
all of his energies and creativity to researching and writing programmed 
materials. At Sullivan Associates, he had carried all administrative responsi- 
bilities, as well as most creative functions. By the end of the 1960's, 
Sullivan was tired and realized that he had done nothing but work for 15 
years. Yet he did not feel that programmed learning could continue to thrive 
and improve without his constant attention. For months, Sullivan was tortured 
with the decision of whether he should continue devoting his life to his work 
or whether he should remove himself from the forefront of programming activity 
and consequently allow programming to die. 

Even though Miss Buchanan had worked with programming nearly as long as 
he had and had even been personally responsible for the creation of many of 
the reading materials, Sullivan did not see her as a potential administrator. 
Finally, however, he became convinced that his long-time assistant was capable 
of assiuilng both creative and administrative functions of Sullivan Associates. 
a£actly how Sullivan's decision was made is not clear. He reports that Miss 
Buchanan had been telling him for 15 years that she could run things better 
than he could, so he finally decided to let her. Miss Buchanan, on the other 
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handj states that Sullivan talked her Into assuming the new role. In either 
case 5 in June^ 1970, Sullivan stepped down and Miss Buchanan became President 
of Sullivan Associates. 

Miss Buchanan and Dr. Sullivan had done all of the actual writing at 
Sullivan Associates. With hi^ departure, Miss Buchanan hired five girls to 
take over some of the writing duties. They ware Stanford graduates with 
degrees in English. Miss Buchanan directed their work, but allowed them to 
work out their own techniques. As members of the Board, Sullivan and Calvin 
give suggestions, but Miss Buchanan makes the decisions under their advise= 
ment . 

Although 90% of Sullivan Associates* work has been with reading, they 
have recently branched out into mathematics, science, and social studies 
programs as well. Miss Buchanan currently employes a permanent staff of 18 
with about 2S free-lance writers and artists fairly constantly employed. 

She has given up writing herself, and now she turns that responsibility over 
to project directors. Miss Buchanan gives directions, the project directors 
develop plans, then together they refine them. The project director is then 
responsible for executing the development plans with the help of permanent 
and free-lance staff. Miss Buchanan gives personal final approval to every 
aspect of every progr^. 

When asked about the major problems the programming staff has enGOuntered 
since the founding of Sullivan Associates in 1961, Miss Buchanan, without 
hesitation, replied, ’*writer*s cramp.** She does include two other areas as 
sources of major difficulty. The first is the area of personnel, Sullivan 
and Miss Buchanan have found it very difficult to find good people for the 
staff. Writing the programmed materials requires brilliant people who excel 
in their subject area and write well, too. The task also requires hard work 
and imagination. Sullivan and Miss Buchanan trusted only themselves to write 
programmed materials for many years. Since Sullivan’s semi-retirement, 

Miss Buchanan has found ”a couple" of people whom she **nursemaided" Into 
becoming good prograimners . 

The second problem area has been production* With their own orientation 
toward writing, both leaders of Sullivan Associates have had difficulty 
realising the importance of production and devoting enough time in each pro- 
duct’s development schedule to it= Miss Buchanan now recognizes the 
Importance of keeping the creative and production phases of product development 
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separate. Clashes between the creative staff and production staff are coramon, 
but Miss Buchanan is attempting to keep both in balance at Sullivan Associates. 
She currently has her managing editor in charge of production to insure that 
the two phases of development operate harmoniously. 

Formative Evaluation 

The Progranmed Reading series was tested in Mountain View, California, 
in a temporary, dittoed form. Arrangements were made by contacting the 
teachers of Lewis Miller’s children. From 1961 to 1963, the materials were 
used by three regular classes and one remedial class. Mr. Millar lived across 
the street from the Mountain View school, and each morning he would take packs 
of dittoed programs in his child's red wagon across to the pupils. No teacher’s 
guide had yet been prepared, so Miller would give oral or handwrJ.tten instruc- 
tions to the teachers for that day’s lessons. Two tons of ditto paper were 
used during the two years of field testing. 

Miss Buchanan and her assistants knew that her materials were effective 
because of the years of developmental testing at Hollins. Sullivan, however, 
insisted that each program have no greater than a 5% error rate, and he 
d^anded that the newly prepared materials be tested in the classroom to 
Insure that that rate was not exceeded , 

The experimental groups using Progranuned Reading were a first grade 
class of nine girls and 13 boys, a second grade class of six girls and 
seven boys, and a remedial class of second- and third-graders having five 
girls and six boys. The control group was equated with the experimental 
group as far as was possible on the basis of age, sex, IQ, and socio- 
economic background. All of the pupils were from ’’average," middle-class 
communities. Both groups were tested at the beginning of the school year 
with the appropriate Lee-Clark or Gates test, and were retested both in 
January and June. Administered by the regular classroom teachers and in 
cooperation with the school principals, the tests were given under uniform 
and standard procedures. 

Results of the first year of testing showed that the error rata on 
the programmed text was less than three percent and on the accompanying 
tests was less than five per cent. In average total reading grade growth, 
the experimental first grade class gained 2.0 years while the control class 
gained 0.9 years. The experimental second grade class gained 1,4 years, 
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while the control group gained 0.9 years. The remedial class using Programmed 
Reading gained 2.3 years in average total reading grade growth. A more 
detailed analysis of the results is Included in Appendix A. 

While the basic books of Programmed Reading were being field tested, the 
supplementary materials for the series were undergoing development. The 
teacher's manuals. Decoding Kit, Placement Examinations, and Pre-Reading 
materials all were prepared during this period. 

Upon conclusion of the field test of Programmed Reading in 1963, teachers' 
and pupils' comments were considered and the materials underwent a final 
minor revision. Final production of the materials was completed by Sullivan 
Associates, including the typing and art work. 

Aside from the programmed texts of the two basic series, the Sullivan 
materials did not undergo extensive developmental testing. Such trial runs 
had been possible before because of the Carnegie grant, but at Sullivan 
Associates, the sums of money necessary for extensive developmental testing 
were not available. Neither did the staff feel that developmental testing 
was necessary to insure the quality of the supplementary products. The 
Sullivan programming techniques had been thoroughly refined at Hollins; .hey 
were known to be effective. The only new eluent to be introduced by any 
individual product was the specific content, and since that content was under 
the strict supervision of Sullivan, they had little fear that the supplementary 
materials would be unsuccessful. Any minor problems about stories or charac- 
ters the children did not like were found during the field tests. 

All of the materials were field tested, but to varying degrees. Field 
test contacts were generally made through personal contacts . \ After a few 
teachers had used Sullivan materials, their reputation spread, and teachers 
requested the opportunity to try out the materials. Schools in the San 
Francisco area were the predominant field test sites. 

Field testa were generally rather informal; staff members would deliver 
the materials to be tested and offer general instruction, then leave the 
materials with, the teacher. The teacher would observe the pupils as they 
used the materials, A staff member would then return to the class, record 
the teacher's observations and comments, and ask the children for their 
response to the materials. Pupil and teacher responses, as well as error 
rates recorded where possible, were then used to revise the materials for 
the published version. 
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SUMMATIVE EVALUATION 



Such a large number of field tests have been carried out by independent 
school districts and researchers that discussing all or even collecting their 
results is beyond the scope of this report. Many of these field tests have 
been conducted completely Independent from Sullivan Associates. 

A few representative field tests are described below. The total number 
of field tests with positive and negative outcomes is not knowni but since 
orders for Sullivan reading materials continue to increasej and since large 
orders are being renewed rather than cancelled, it may be assumed that results 
of using the materials are favorable more often than they are unfavorable. 



Colorado Springs School District 11 

The school district in Colorado Springs, Colorado, used the Programmed 
Reading materials during 1965-66. Mr. Lew Miller, then a consultant for 
McGraw-Hill, and Mr. Lyre Patterson, a McGraw— Hill sales representative 
^^^hiated the field test with Dr . William Liddle of the district. It was 
dscided that one first grade class in each of four schools would use the 
Progranmed Reading materials exclusively. A control group In each school 
was set up to use the McKee basal reading series from Houghton Mifflin. 

The four schools represented areas of high socioeconomic, an upper— middle , 
low-middle, and low socioeconomic status. The children's IQ's ranged from 
very low to very high. Miller acted as a consultant in getting the teachers 
started. Sullivan and several other people also assisted as consultants. 

The experimental group was considerably disadvantaged because of a 
l3.rge turnover rate in the schools. All new first grade pupils entering 
the district during the year were placed in Programnied Reading classes. 

Pupils in the experimental group numbered 114; in the control group, there 
were 113 pupils. Sixty-two girls and 32 boys made up the experimental group. 
The control group had a similar composition. Instruction was given for 180 
days, and the teachers had 4, 9, 13, and 14 years of experience, respectively. 

The experjmiental and control groups were vary similar in levels of 
reading readiness and in IQ at the beginning of the year. At the end of 
the year, the Metropolitan Achievement Priniary I Battery was administered 
to both groups. Combining the four experimental classes and the four con- 
trol classes, statistically significant differences were found in Word 



Knowledge (p<*02)s Word Discrimination (p<.01) and Total Reading (p<.01)j 
all in favor of the experimental group. Great enthusiasm was shown by 
teachers, parents, and children alike toward the materials. 

' San Francisco Unified School District 

During the 1968-69 school year, the San Francisco Unified School District 
measured the performance of 1,276 Project RF^^^pupils. Twenty^SByen schools 
participated, all but six of which were located in poverty areas*. Pupils 
were pretested in May, 1968, and poattested in May, 1969, with the Stanford 
Reading Test, Each child used the materials two or three times a week 
throughout the year* 

Dr* Robert Jenkins, then Superintendent of the San Francisco Unified 

■ . t ' - 

School Distriet (September, 1969) commented ori the results of the pre^ and 
posttesting: j . 

^ j * ■ 

Stanford Achievement Test is not an instructional 
test reflecting local goals, but rather it is a status 
test, culture-bound, relying , on ^eKperiences children 
hkve hkd* Correct responses are empirical: they reveal 

Experiences childrett 'Aave undergone and what has 
happened in the^ past under , conditions of .the .past- * .In 
the interpretation of such test data, it Ig important 
to focus upon tHe*uppe* quarter of these 'pupils who 
tested at or very c loce to grade level on the standardized 
test. . . 

” These children have more successfu].ly realized their 
abilities and capitalized upon previous eKperiences to 
“ 'the point where, by test standards, they are performing 
satisfactorily. This quarter of the San Francisco pupil 
population is nonetheless urban in background; the 
reader is reminded, as explained in prior test reports, 
that the test norming process did not include many large 
city pupils [p. 6]. 



Summarizing the results of the project, Jenkins reported (1969) that 
ten of the schools showed one year or more of reading growth, and the other 
tdri “Showed eight or nine months of reading improvement* Further, 43% of 
all pupils in the district in grades H3 through H6 made month^per -month 
gains. Sixty-three percent, however, of a comparable group of Project 
READ students made the s^e month-per^month galiis. These results compare 
most favorably to earlier tasting before Project READ installation that 
showed month-* par -month loss of reading skill (pupils ware falling further 
behind actual grade placement levels) . 
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The Metropolitan Readiness Test was administered to all San Francisco 
public school pupils at the end of kindergarten or the beginning of grade 
one. Using this test, the readlnass status of a pupil can be compared to 
the national norms In which the average pupil would be ranked at the 50th 
percentile, the highest scoring quarter of pupils at or above the 75th per- 
centile, and the lowest quarter at or below the 25th percentile. 

In 1966-67, Project READ had not yet been installed in the schools. 

In March, April, and May of 1968, Project READ had an eight-week trial run 
in 31 schools. During 1968-69, Project READ had a full year of operation. 

Results of the three years of testing with the MRT showed the follow- 
ing; 

1. Readings status of all pupils in Project READ Increased 
markedly during the two years of the program at the 
kindergarten level. For all 31 project schools, the 
25th percentile status children increased by 11 points 
on the national percentile scale, the 50th percentile by 
15 points, and the 75th percentile by 14 points. 

2. improvement in eight bilingual schools (those schools 
vlng a majority of their pupils with Chinese or Spanish 

as a native tongue) was about 50 % greater than in the 
bilingual schools: 23 to 15 points of increase at the 

75th percentile, 21 to 13 points at the 50th percentile, 
and 13 to 8 points at the 25th percentile. 

3. In the dlsadvantaied schools (25 of the 31), improvement 
seaned to have been greatest for the 75th percentile and 
least for the 25th percentile children. A reversal of 
this finding was observed for the six advantaged schools. 

Complete data are provided in Appendix B. 



Vicksburg Si^Maar Raadlng Program 

During the summer of 1968, the Vicksburg, Mississippi, Municipal 
Separate School District offered an intensive remedial reading program, 
using the Programmed Reading materials. The program was funded by Title I, 
ESEA. Two full time consultants from BRL assisted In Impl^enting the pro- 
gram, which was to run four hours a day for seven weeks. A pre-service 
workshop totaling about six hours of Instruction was conducted by the BRL 
consultants. The staff was Introduced to the materials, instructed in 



placement of pupils at the proper reading level, and in methods of present^ 
ing the materials to the students. 

All pupils in grades 1 through 6 with below grade level reading ability 
were invited to attend. Of the slightly more than 700 pupils registered, 

83% were black and 17% white. Approximately 50% of the pupils were from 
low-income homes. Each class, consisting of about 26 pupils, was led by a 
team of one white and one black teacher. Students were placed with others 
of their general ability after taking the Gates-MacGlnltie Reading Test for 
vocabulary and comprehension levels. This test was given the first day of 
class . 

Pupils spent approximately 3^ hours each day using the programmed texts, 
interspersed with periods for supervised activities , free reading, listening 
to taped stories, aad refreshments. Teachers attended two^hour workshop 
sessions with the consultants each afternoon to review the morning’s activities, 
prepare instructional materials, and review methods of presentation. Teach- 
ing techniques and examples were illustrated on video tape. 

Of the 60% of the pupils who were both pre- and postteste 1 with the 
Gates-MacGinitie Reading Test, 59% showed measurable achievement in vocabu- 
lary and 54% showed positive achievement in comprehension. Table 1 shows 
the percentage of students at each level of achievement (Fairley, 1968). 

Table 1 



Results of Vicksburg Surmer Reading Program 
Using Programed Reading Materials 



Achiavement 


Vocabulary 


Comprehension 




In 


Years 


Percentage 


Per can tage 




+ 


.1 


9.5% 


7.3% 




+ 


.2 


8.8% 


7.5% 




+ 


.3 


6.3% 


7.8% 




+ 


.4 


3.7% 


3.2% 




+ 


.5 


3.9% 


5.8% 




+ 


. 6 


5.3% 


3.6% 




+ 


.7 


3.9% 


1.5% 




+ 


.8 


5.4% 


3.4% 




■f 


.9 


2.2% 


2.7% 




+ 


1.0 & over 


10.0% 


10.1% 
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The school district reports that there was evidence of high interest 
by the students in the progranit Teachers often related incidents of pupils 
wanting to continue work In the programmed texts rather than take part in 
play activities. 



Belcher, Louisiana 

During the 1969-70 school year, 1,290 children In grades K through 12 
In Belcher, Louisiana, participated in Project READ. Not all students and 
teachers participated the entire time because of transfers. Stanford 
Achievement Tests and California Achievement Tests were given as pre- and 
posttests. The Lee-Clark Reading Readiness Test and Sullivan Reading 
Placement Tests were also used. A total of 69 teachers were trained by 
BRL consultants, and in-service consultations were also utilized. 

The development and growth of language arts skills was evaluated by 
the tests already mentioned, as well as by observance of pupil work habits, 
of expressions of increased academic security, and of the extent of pupils' 
conmand of skills. Participating teachers conmented that pupils' attitudes 
toward learning had improved and positive self-concepts had been developed 
and/or strengthened, they became more Independent in their work habits, 
potential dropouts increased their attendance, and pupils who had been pre- 
viously withdrawn now participated voluntarily. 

Data collected for the October 28 to February 17 semester is shown in 
Table 2 (Rollins, 1970). Average gain in grade placement was 6 months, 
compared to a 3.2 month expected gain. Figures in the table repress:- 1 vaan 
grade placements . 

Kern County Joint Union High School District 

This district in Bakersfield, California, used Sullivan programmed 
reading materials during 1966 and 1967 with 50 severely disadvantaged ninth 
grade students whose reading abilities ranged from the fifth grade level to 
the non-reading level. Since considerable evidence has already been indi- 
cated resulting from other field tests using a much greater number of 
students, the data concerning Gognitive gain will not be constdered here. 
However, Dr. Dave Martin of the University of Southern California studied 
the effective results of the progranmed materials. 
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Table 2 



Results of Belcher j Louisiana, Evaluation of Project READ 



(P <.Q1) 



i 

Grade 


No. 

Tested 


LEE-CLAEK 

READING 

READINESS 


STANFORD ACHIEVEMENT 
TEST 

Pre=Test Post““Test 


CALIFORNIA ACHIEVEMENT 
TEST 

Pre-Test Post-Test 


K 














1 


76 


0,7 




1.5 






2 


85 




1.4 


1.7 






3 


86 




1.5 


2.2 






4 


83 








3.0 


3.7 


5 


93 




2.9 


3.9 






6 


96 








4.3 


4- 7 


7 


104 




4.1 


4.3 






8 


101 








5 • 6 


5.8 


9 


121 




4.9 


6.1 






10 


110 




f 




7.5 


8.1 


11 


100 




5,2 


6.1 






12 


100 








8.1 


8.4 
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Forty'-five of the students in the program were interviewed by Dr, Martin 
in April, 1967. The interview consisted of ten open-ended items. Interviews 
were taped and the tapes were coded and analyzed. 

The first question was designed to elicit expressions of general affect, 
^"ifty-slx per cent of all pupils responded with *'Unqualif led Liking" of the 
program, 42% with "Qualified Liking." and 2% ( 1 pupil) with "Dislike." 

The second question was designed to find the extent to which pupils 
thought the class helped them. Eighty— seven per cent responded with "Helped 
Greatly," 9% with "Helped Somewhat," and 4% with "Doubtful or None." 

third question was, Wlrat are some of the things you enjoy doing 
most in this class?" Responses are listed below: 



Sullivan Workbook 


48% 


Sullivari Reader 


9% 


Games 


18% 


Plays 


4% 


Discussion 


7% 


Tapes 


2% 


Everything 


2% 


Nothing 


10% 



Another question was, "How do you think the other students like the 
class?" Responses were summarized: "General Liking" received a 56% response; 

"Ambiguous or Don't Know" received a 33% response; and "General Dislike" 
received a 9% response. 



DIFFUSION 

Diffusion of the Sullivan reading materials is generally the responsi- 
bility of Behavioral Research Laboratories under Allen Calvin, BRL owns 
all of the Sullivan programmed materials and publishes them. McGraw-Hill 
Book Company does market the Programmed Reading series, however. 

In 1962, while Programmed Reading was undergoing field tests in 
Mountain View, California, Mr. Ken Ziegler of McGraw-Hill evinced interest 
In doing the marketing of that series. Sullivan, who then held the copy- 
right, signed a contract to that effect with the Webster Division of 
McGraw-Hill, with Sullivan and the company each receiving half the profits. 
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Later, Sullivan traded his copyright and his share to BRL in return for 
shares of stock. McGraw-Hill began marketing Progranimed Reading in 1963 but 
according to Lew Miller, their efforts were not very successful. Only about 
$187,000 was received the first year. Mh. Miller left Sullivan Associates 
and joined McGraw-Hill the following year to instigate better marketing 
procedures. By 1970, $10,000,000 in Programmed Reading (revised edition) 



had been sold • 

In 1965, BRL founded its own sales force, to handle marketing of the 
rest of the progratnmed material developed by their own group and by 
Sullivan Associates as well. In 1966, the Sullivan Reading Program that 
Sullivan had personally authored was published by BRL, who also had handled 

the production phase. 

During the early days of dissemination of the programmed materials. 



McGraw-Hill and BRL had to contact school districts to try to sell the 



programs 

received 



Use of the Sullivan materials has had such a great impact and has 
so much publicity, however, that by the lata 1960's, most sales 



were user initiated. 

In 1968, BRL developed the teacher-training and consultation services 
to accompany Sullivan Associates’ reading materials and began marketing 
Project READ, which now accounts for the great majority of reading program 
sales. If a school district can place an order for $40,000 or more for 
BRL/Sullivan reading materials, the district is eligible for Project READ, 
which is currently in use In at least 100 large urban school districts. 
Calvin estimates that the relationship of materials sold under Project READ 
to materials sold individually as 5 to 2. He also estimates that under 
Project READ, the cost per student Is less ($.20) than without the added 
project services. Small school districts, unfortunately, have difficulty 

xnBstl-iig the 040^000 miuxmuni oirdex' Ximlt . 

Project LEARM Is a newer system offered by BRL that uses Sullivan 
materials. This project includes BRL assumption of school management respon 
slbility. In return, they guarantee a certain level of student academic 
improvement. If students do not show promised improvement, the school 

district does not have to pay the fu3.1 cost. 

The newest diffusion technique utilized by BRL to disseminate Sullivan 
reading materials is the opening of Sullivan Reading Centers m cities 
across the country. According to 27 different criteria, BRL selects a 
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possible site for a reading center, then works with the approval of a local 
school district to set it up. The 27 criteria are trade secrets according 
to Calvin. Directors of the centers are carefully selected | to this point, 
they have all been teachers who have received special training. Parapro- 

fessionals, generally graduate students or teacher trainees, also staff the 
centers , 

Sullivan Reading Centers are open to anyone who wishes to Improve his 
reading skills. Newspaper, magazine, and billboard advertisements, and 
teacher referrals are used to make poor readers and parents of poor readers 
aware of the service. Each potential customer is given a battery of diag- 
nostic reading tests to determine the nature of his reading problem. Most 
pupils go to the center after school, but special arrangements can be made 
for a child to go to the reading center instead of his regular reading c .ass 
during school hours. Sessions last at least four weeks to give each child a 
chance to realize noticeable Improvement. Pupils may attend from two to 
five times each week for one hour each day. For the fee of $5 per hour, the 
pupil has his reading problems diagnosed, books assigned that he may keep, 
and professional and paraprof esslonal tutoring whenever he requires it. 

ADOPTION 

Extent of Product Use 

Allen Calvin estimates that approximately 5 million people are using 
Sullivan programmed reading materials. At least 100 large urban school 
districts use Project READ. There are 17 Sullivan Reading Centers, with 
many more planned for the future, 

Calvin describea programmed materials as the fastest growing, most 
widely accepted method of innovative teaching. So many schools use the 
reading materials, that Calvin feels it is impossible to characterize the 
majority of users; all kinds of schools use them. Generally, the Sullivan 
Reading Program is used more by inner-city schools while che Prograiraned 
Reading distributed by McGraw-Hill is used more by schools with pupils less 
likely to have serious reading problems. Since sales are growing tremendously 
the big project orders are renewed and new cities are placing orders, Calvin 
assumes that users are pleased with the materials. 
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Ins t alia t ion Procedures 



a school simply orders the materials, teachers rely on the teacher's 
for injformatlon on impleiaeritation of the program. The only adapta- 
a standard classroom situation that might be necessary is rearrange-^ 
ment ox the furniture to accommodate the different simultaneous activities 
of pupils working individually. 

Since the role of the teacher in a Sullivan classroom is probably a new 
one for most teachers, teacher training is most helpful and is provided when 
the materials are purchased in a project. When no teacher training or con- 
sultation is provided, teachers must rely on the teacher's manuals, but these 
instruments are carefully designed to give the teacher as much information 
and advice as might be needed. Teacher training provided with Project READ 
includes presentation of films, role^^playing , and workshops. The evaluation 
of these teacher training procedures also falls within Calvin’s broad cate= 
gory of "trade secrets." 

Extra staff within a classroom using Sullivan reading materials is not 
required, but any paraprof essionals in the classroom can be helpful to give 
P^pffs with individual problems more thorough assistance. 

Because of the wide variety of reading materials available as part of 
the Sullivan program, considerable product modification is possible* Any 
combination of the materials is purchasable* Optimal success is, however, 
linked with the properly sequential use of the major materials. 

In the early days of progranmied materials, considerable public relations 
work had to be done before the materials could be sold* Teachers were very 
skeptical and suspicious of the new teaching method; they feared that their 
own role would be diminished if programmed texts could teach so well* After 
years of publicity and good results, however, programmed materials are no 
longer seen as threatening and they are easily acceptable to teachers and 
administrators as a more effective substitute for standard reading programs* 



Success of Installation Procedur es 

Sullivan reading materials seem to be easily installed in nearly every 
type of school. If teachers follow the suggestions given by BRL and dis- 
cussed in the Description of Materials section of this report, instair.at ion 
should not be problematic* Publicity In magazines and newspapers, ranging 
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from the Newark Evening News to the Congressional Record ^ overwhelmingly 
records the enthusiastic response of teachers ^ parents ^ and pupils who have 
had experience with the Sullivan materials, 

COSTS 

Because early development of the reading research is so intertwined with 
the research for other subjects, it is impossible to determine the costs of 
developing the reading materials. The developers themselves do not really 
know what has bean spent; their early concern was academic and not financial. 
Three and one— half to four million dollars was spent in research and develop- 
ment at liollins. Perhaps one-third to one=half of that was for reading. But 
millioiis of dollars have been spent on every phase of development. When 
Sullivan and Calvin founded Sullivan Associates and Behavioral kesearch 
Laboratories s they started a multi-million dollar educational empire. BRL 
lists total sales for 1970 as more than $10 million, but that, of course, 
includes many other materials besides reading programs. 

FUTURE OF THE PRODUCT 

When Sullivan realized the effectiveness of programmed materials, he 
and his colleagues believed that within decades, education would be converted 
almost entirely to progranmiad learning. Their enthusiasm and expectations 
have cooled somewhat, but sales are steadily increasing and more and more 
subjects are being programmed. The reading programs are being continually 
supplemented, and will undergo revision as it becomes necessary. 

When progr aimn ing was new, voices from many places were raised to say 
they were going to do programming, too. Some "programs” were published that 
are not really progrms, as far as Sullivan is concerned. They are simple 
repetitions of materials, formated in boxes, with blanks and questions 
inBarted, The learning theory principles of stimulus and response are 
missing. If another group were to begin writing "real" programmed materials, 
they would have to begin where Sullivan began in the 1950’ s. He feels that 
no one else would probably have the required intelligence or drive to be as 
successful as his group has been. 

Sullivan Ajsoclates and Behavioral Research Laboratories do not intend 
to divulge their myriad trade secrets either. They have worked very hard for 
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a long time and they now have every intention of reaping the generous rewards 
of the prograBimlng business, Sullivan programmed materials published by BRL 
currently account for about 95% of the programmed materials on the market* 



Sullivan emphasizes that it is virtually impossible to enumerate the 
critical decisions that ware made during the decade and a half covered by 
this report. Thousands of decisions were made^ some critical, but most rather 
insignificant. They did all combine, however, to mold the products of the 
Sullivan prograinmed materials into the form in which they exist today* 

Listed below are a few of the decisions that seem, in hindsight, to have 
been major factors in the development of Sullivan reading materials* 

Decision 1; To Work with Programined Lessons 

Perhaps the most momentous decision involved in the development of the 
Sullivan reading materials was Sullivan's initial decision to work with pro-“ 
grairrmed materials. That the linguist would work with prog ramming seems to 
have been predetermined rather than decided. His career had lacked a focus 
and suddenly, upon learning of Skinner's early attempts, he knew that he 
would program. 

The development of the reading materials was not a case of wanting to 
improve reading instruction and then seeking the means to do sc* It was 
instead a case of discovering a valuable teaching tool and seeking a deserv= 
ing subject to teach with it* Had Sullivan (and others) not taken up the 
programming aegis in 1956 , the teaching of reading might be plodding along 
in the tradition of centuries. 

Decision 2 ; To Program Reading Materials 

Sullivan's early programming efforts were In the area of modern languages* 
His decision to program reading materials came somewhat later in a fit of 
frustration* He had striven diligently to write programs in modern languages 
that would yield a very low error rate. Yet high school students simply 
could not master his programs. Convinced that his programs were not at 
fault, he concluded that the students were to bl^ae; they could not read 
well enough to follow his programs* To keep the students from making his 
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error rates look bad^ and also because he realised that there was something 
quite ineffective with current reading instruction, Sullivan decided to turn 
his programming efforts to the subject of reading. 

Decision 3: To Provide for Teacher Training and Supplementary Materials 

Sullivan and Calvin decided in the mid“1950*s that development of 
prograruned materials could not rely solely on the writing of programs. They 
realized that programming was a large-scale innovation and that it must be 
treated as such. A new method of teaching required a new teaching environ- 
ment. Consequently , they accepted the task of not only writing programmed 
materials, but of restrw; ^turing the world of the classroom through teacher 
training and supplemanf ary non— programmed materials. This decision eventually 
evolved into the complex systems approach represented by Project READ and 
Project LEARN. 

Decision 4: To Write Prog7:ani Materials Himself 

Sullivan decided to write programs himself, despite the incredible 
tadiousnass of that task, rather than merely direct other programmers. This 
decision seems to have stemmed from his lack of faith in other people -s 
ability. Sullivan found that most other people who tried programming simply 
could not do it. It required an absence of ego involvement, since each pro- 
gram was inevitably criticised through revision after revision. Sullivan 
discovered that few people besides himself could spend the long hours writing 
and rewriting that programming required. Consequently, he had the personal 
responsibility of writing and rewriting nearly everything that was done. Had 
he not reserved so much responsibility for himself, Sullivan most likely 
would not have been so e^austed with programming in 1970 that he had to 
face the agonising decision of whether or not to retire from active partici— 
pation. 



Peciaion 5t To Discontinue the Use of Teaching Machines 

Sullivan in 1957 decided to discontinue use of the tape recorders and 
more complex machines in association with the programmed taxtSi. Sullivan 
states that reliance on machines had been unwise, but that it was under^ 
standabla. First of all, ii* F* Skinner had been machine-oriented, and 
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Sullivan had inherited that orientation from him along with the concept of 
programming ■ Secondly , Sullivan’s own background had been with teaching 
machines before prograimnlng had started. NonethalesSj they discovered that 
the machines were too axpensivSj that they broke down too easily and were 
often inoperable, and that each progr^mi revision entailed revision of tapes 
as well^ That the machines were merely making programming more complex 
finally occurred to him. Discontinuing their use was a major decision; 
every program had to be entirely rewritten and progranmiing embarked on a 
new path. Had this decision not been made, however ^ it seems likely that 
progr^mned materials would never have achieved such comaon usage, for the 
cost and complexity of operation would have been prohibitive. 

Decision 6; To Utilize "Chalnlns" Techniques 

In 1958, Sullivan initiated another major format change with his inven- 
tion of ’’chaining.” Children were becoming easily bored with the tremendously 
repetitive format that Skinner had used and Sullivan had accepted, and one of 
Sullivan’s chief goals was to keep children enthused and interested. To 
alleviate this problem, he decided not to repeat blocks of information in 
each frame, but to allow each frame to build on information presented earlier - 
Consequently, children could proceed through a program much more quickly. 

This decision, too, probably was vital to the success of the program. No 
matter how valid the theory behind programming, students would not have 
enjoyed working with it if Sullivan had persisted in presenting the same 
information over and over. 

Decisi on 7 i To Use a Vertical Format 

Ar^.othar major format change came about as a result of Sullivan’s 
decision to use a vertical format rather than a horizontal one, as has been 
explained under Actual Development Procedures. This decision was most 
likely not of as great a consequence as the two previous decisions listed. 



Decision 8; To Leave Hollins 

In 1961, Sullivan, Calvin, Miss Buchanan, and a number of other Hollins 
staff members decided to leave Hollins and move to California as employees 
of Encyclopedia Britannlca Films. They saw the move as a means of escaping 
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the interferanca of the Hollins administration with their work. Relations 
with EBF had up to that time been good and they anticipated more freedom to 
carry on their research as they wished* Deciding to work for EBF was an 
immediate fiasco. They found that EBF intended to impose even more restrict 
tions than Hollins College had, and that the company’s commercial orienta^ 
tion just could not coexist peaceably with their own academic leanings. 
However, leaving Hollins did break the programming staff out of their rut 
and did in the end lead them to set up a workable situation on their own. 

Decision 9: To Form Sullivan Associates and BRL 

The decision by Calvin and Sullivan in 1961 to set up thair own companies 
was perhaps an inevitable one. They had tried and failed to work in both a 
university setting and a commercial setting. The only perfect setting for 
their unique operation would have to be one of their own making. It was this 
decision that yielded the very successful blend of research, development, and 
coimaercialism that Gharacteriges the programming operation today* And, of 
course, had they simply given up programming at that point, this teaching 
method might never have progressed beyond the basic research stage. 

Decision 10: To Revise the Prograsmied Reading Series 

The decision to revise the Frogrammied Reading series was made in order 
to meet California requirements for adoption as a state-approved text. In 
1966, California turned down Progranmaed Reading, but Sullivan Associates 
felt that adoption was an important enough goal to spend 18 months revising 
and improving the series. The revised edition met state standards, and 
considerable salsa have resulted. 

Decision 11: To Retire as President of Sullivan Associates 

Sullivan’s decision to retire from the presidency of Sullivan Associates 
was the result of considerable anguish. He felt he could not continue working 
at the extr^e pace he had kept up for 15 years. Yet he feared that pro- 
gramming would die without his personal constant attention. He seemingly 
faced either his own collapse or the collapse of his lifci’s work. He finally 
was convinced, however, th.at Hiss Cynthia D. Buchanan was capable of carry- 
Ing on both administrative and creative functions of Sullivan Associates - 
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The company seems as successfully productive as everj and Sullivan seems 
well adjusted to the more leisurely pace of his current projects. 
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APPENDIX A 



RESULTS OF MOUNTAIN VIEW FIELD TEST OF PROGiU^FJD READING 



Data are presented under the fQllowing headings: 



Table A-1 Data for ExperimentHl Groups Using Programmed 
Reading and for Equated Controls 

Table A-2 Individual Reading Grade Growth for Students 
Using Prograirimed Reading 

Figure A-^1 Evaluation of Progranmied Reading With Remedial 
Remedial Group vs. National Norm 



Figure A^2 Evaluation of Programmed Reading in Normal 

Second Grade — Experimental Group vs . Control 
and National Norm 

Figure A--3 Evaluation of Programmed Reading in Normal 
First Grade-^^Experimental Group vs. Control 
and National Norm 





Table A~1 

Data for Experimental Groups Using Programmed Reading and for Equated Controls 
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groups equated for I.Q.; scores not available from school. 



Table A-2 



Individual Reading Grade Growth for Students Using Prograinnied Reading 



Sept. j 1962 - June, 1963 
22 First Grade Students 



Rank Order: 


Reading 


Grade 


Placement 


Total 

Reading 


I.Q. Distribution 


Lowest to 
Highest 


Sept . “ 
Oct . 


Jan - 


June 


Grade 

Growth 


I.Q. 

Interval 


Frequency 


1 


1.3 


2.00 


2.48 


1.18 


80-84 


1 


2 


1. 8 


2.00 


3.03 


1.23 


85-89 


0 


3 


1.6 


2.23 


2.92 


1.32 


90-94 


0 


4 


1.5 


2.18 


2.88 


1.33 


95-99 


0 


5 


1.8 


2.70 


3.45 


1.65 


100-104 


1 


6 


1.8 


2.21 


3.47 


1.67 


105-109 


5 


7 


1.5 


2.26 


3. 20 


1.70 


110-114 


2 


8 


1.6 


2.51 


3.40 


1.80 


115-119 


3 


9 


1.7 


2.33 


3.53 


1.83 


120-124 


1 


10 


1.5 


2.43 


3. 36 


1.86 


125-129 


5 


11 


1.1 


2.32 


3.03 


1.93 


130-134 


4 


12 


1.8 


2.35 


3. 75 


1.95 


135-139 


0 


13 


1.8 


2.53 


3.87 


2.07 


140-144 


0 


14 


1.4 


2.31 


3.50 


2.10 


145-149 


0 


15 


1.8 


2, 89 


3.95 


2.15 


Mean I.Q 


. - 117 


16 


1.6 


2.54 


3.77 


2.17 







17 


1.8 


2.53 


4.15 


2.35 


18 


1.4 


2.42 


3. 77 


2.37 


19 


1.4 


2.42 


3.93 


2.53 


20 


1.8 


2.84 


4.30 


2.60 


21 


1.8 


2.56 


4.60 


2.80 


22 


1.8 


3.01 


4.85 


3.05 


Mean 


1.6 


2.48 


3.60 


2.00 
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Figure A-1 



Evaluation of Programmed Reading ViJith Remedials 
Remedial Group vs. National Norm 




Date of Testing 





54 



Mean Reading Grade Score 



Figure A-2 



Evaluation of Programmed Reading in Normal Second Grade 
Experimental Group vs. Control and National Norm 
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Mean Reading Grade Score 



Figure A- 3 

Evaluation of Programmed Reading in Normal First Grade 
Experimental Group vs. Control and National Norm 
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APPENDIX B 



RESULTS OF THE SAN FRANCISCO FIELD TEST OF PROJECT READ 

The Grade Sequence coluian Is listed In terms of semesters | HI indicates 
the second semester of grade 1, L5 indicates the first semester of grade 5. 

Test data are presented according to three analyses. 

The status (colunin A) is represented by the national— norm grade equi— 
talents at the 75th, 50th, and 25th percentiles for pre— and posttest score 
distributions . 

The gain (column B) batween pre- sind posttest scores was calculated for 
each pupil. The percent of pupils who recorded month-fcr-month gain is 
recorded. (Harold L. Weeks, Director of the Division of Research and Program 
Evaluation for the project, points out that generally pupils characterized 
by poverty do not demonstrate month-for-month gain, but fall farther behind 
grade level as they proceed . ) 

The relationship (column C) of each pupil's reading score to actual 
plSiCfflQent (A.G.P.^ at the time of testing was classified ass (1) at 
or above grade placement (AGP/+) • (2) . 1 to .9 year below actual grade 
placement (-.1/.9); or (3) one year or more below actual grade placement 
(~1.0/+). 
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Standardized Test Evaluation o£ Project READ During School Year 1968—1969 



I. PRE- AND POST-TEST STATUS AND PUPIL SCORE CHANGE FOR PROJECT READ PARTICIPANTS 
IN SIX ELMENTARY GRi^DE SEQUENCES: STANFORD READING TEST, TOTAL READING 



Grade Seciuence 
Test Month 4 Year 
Teat Level & Form 
(A.G.P. At Testing) 
Pre-Test Post-Test 


Elap- 

sed 

Time 

in 

Years 


Number of 
Pu- Sch- 
plls ools 


Per 

cen- 

tila 


(A) 

Status of Group: 
Grade Equivalent 


1 ! (B) 

1 Pupils 
Gains - 
% Mon,^ 
1 for-Mor 


Scale 
in Re 


(C) 

Per Cent 
of Pupils 


Pre- Post- 
Test Test Dlff. 


A.G.P. 
. (Years) 


Pre-" Post- 
Tast Test Dlf 


O’”! 

w ai m 

VO 




327 18 


75 » 

50® 

25* 


1.7 

1.6 

1.4 




AGP/+ 

-. 1/.9 

• 0 / •¥ 


22 

78 

0 


m. 

May *68 U^'69 

P I W P II w 

0>1 /o Q'l 

\A,o/ K^‘ 0 ) 


1.0 


354 19 


75m 

50 «< 

25ft 


1.6 2.8 + 1.2 

1.5 2.0 +0.5 i 

1.4 1.8 + 0.4 


30 ^ 


AGP/+ 

-.I/.9 

- 1 . 0 /+ 


13 26 +ij 

87 38 - 4 £ 

0 36 +3^ 


H 2 H 3 

! 4 ay *68 Slay *69 

P II W P II X 

(2.8) (3.8) 


1.0 


359 18 


75 * 


2.7 3,7 +1.0 

2.1 2.8 +0,7 

1.8 2.3 +Q .5 




AGP/+ 

~.l /.9 

- 1 . 0 /+ 


23 24 + J 

43 25 -If 

34 51 +1’/ 


10 h 4 

May *68 May *69 

P II Z Int.IZ 

( 3 . 8 ) ( 4 . 8 ) 


1.0 


108 6 


75 * 

50^ 

25 ® 


3.4 4.8 + 1.4 

2.6 3.7 +1.1 

2,0 3.0 +1.0 


6 kf$ 


AGP/+ 

-. 1/.9 

- 1 . 0 /+ 


14 27 +1] 

27 19 - 6 

59 54 - S 


l 4 L 5 

May *68 Jfay *69 

P 11 W Int.JZ 

(^.3) ( 5 . 3 ) 


1.0 


43 4 


75 * 

50 « 

25* 


3.2 4.5 +1.3 

2.7 3.7 +1.0 

2.2 3.0 +0.8 


56^ 


AGP/+ 

-.1/.9 

-1.0/+ 


2 16 +li 

i4 9 - t 

84 75 -5 


h 4 H 5 

May '68 May '69 

P II Y Int.IX 

( 4 . 8 ) ( 5 . 8 ) 


1.0 


57 3 


75* 

50* 

25m 


3.8 5.8 +2.0 

3.2 4.5 +1.3 

2.7 3.7 +1.0 1 


15% 


AGP/+ 

-.1/.9 

-1.0/+ 


18 24 + e 

5 18 +lj 

77 58 -IS 


l 6 ' H 6 

Oc t ’ 68 June ' 69 

Int.IIff Int.IIY 

(6.1) (6.9) 


0.8 


375 9 


75 * 

50* 

25® 


5.2 6.5 +1.3 

4.2 5.1 +0.9 

3.5 4.2 +0.7 


62 % 


AGP/+ 

-.1/.9 

-1.0/+ 


10 20 +1C 

15 13 - S 

75 67 - e 


Six Grade 
Sequences 


1296* 

27 




U6% 


4 GP/+ 

-.1/.9 

-1.0/+ 


15 23 + e: 

43 24 -IS 

42 53 +13= 



*Th±a total does not include -ttie 327 IP’ade m pupils for whom no gain scores were available J 
in the absence of a pre-teat. 
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Standardized Test Evaluation of Project READ During School Year 1968-1969 

II. ms- AND POST-TEST STATUS AND HJPIL SCCeE CHAiTOE FOR PROJECT READ PARTICIPANTS 
IN SIX ELa®JTARY (MADE SEQUEJICES; STANP®D READDJG TEST, WORD MP^ANPJG 



Grade Sequence 
'eat Month & Year 
'eat Ijevel & Form 
A.G.P, At Testing) 
re-Taa t Po at-Te st 

HI 

May '69 
P I W 
( 1 . 8 ) 



HI 

ay *68 
I W 
L.8) 




;l4 

|y*68 
> II W 

C-3) 



|y'68 
III Y 
| 8 ) 



£ 

;»68 

i.liw 

1 ) 



I%y*69 
P II w 
( 2 . 8 ) 



10 

May '69 

P II X 
( 3 . 8 ) 



'h4 

toy *69 

Tnt.EC 

( 4 . 8 ) 



L5 

toy '69 

Int.DC 

( 5 . 3 ) 



H 5 

toy *69 

Int.IX 

( 5 . 8 ) 



h6 

June *69 
Int.IlY 
( 6 . 9 ) 



Six Grade 
Sequences 



Elap- 

sed 

Time 

in 

Years 



Nu^er of 
Pu- Sch. 
pils 00 I 3 

327 18 



1.0 



1.0 



1.0 



1.0 



1.0 



19 



359 18 



108 6 



43 4 



57 3 



0.8 



375 9 



1296* 



27 



Per 

cen 

ti]^ 

75th 

50 ® 

pctt 



75*^ 

50 a 

25 * 



75th 

50 m 

25 th 



75tt 

50 ® 



75 ® 

50® 

25 ® 



75 ® 

50® 

25® 



75 ® 

50® 

25th 



(A) 

Status of Group! 
Grade Equivalent 



Pre- 

Test 



Post- 

Test 

1.8 

1.5 

1.3 



Diff, 



1.7 

1.4 

1.3 



2.8 + 1.1 

2,1 +0.7 

1.8 +0.5 



2.8 

2.1 

1.8 



3.8 

2.9 
2.3 



+ 1.0 

+ 0.8 

+0.5 




4c^ 



38^ 



2.7 



4.7 

3.6 



2.0 3.2 



+ 1.4 

+0.9 

+1,2 



3.3 



5.2 

3.7 

2.5 3.1 



+ 1.9 

+0.9 

+ 0.6 



3.6 
2.9 

2.7 



4.7 

3.6 



+2.3 

+ 1.8 

+0,9 



5.1 

4.1 

3.5 



6.6 

5.1 

4.1 



+1.5 

+ 1.0 

+ 0,6 



58 ^ 



5^ 



70 ^ 



49 .^ 



Scale 
in Re 

A.G.P. 

( Years ) 

AGP/+ 

-.I/.9 

- 1 . 0 /+ 



(C) 

Per Cent 
of Pupils 



'+ 
-. 1/.9 
- 1 . 0 /+ 



Pre- Post- 
Test Test DIfd 



27 

73 

0 



AGP/+ 

-. 1/.9 

- 1 . 0 /+ 



AGP/+ 

-. 1/.9 

- 1 . 0 /+ 



AGP/+ 

-. 1/-9 

- 1 . 0 /+ 



AGP/+ 

-. 1/.9 

- 1 . 0 /+ 



17 25+8 

83 44 -39 

0 31 +31 



25 26+1 

44 26 -18 

31 48 +17 



17 22+5 

27 19-8 

56 59 H- 3 



2 23 +21 

19 12-7 

79 65 -i 4 



AGP/+ 

-. 1/.9 

- 1 . 0 /+ 



AGP/+ 

-. 1/.9 

- 1 . 0 /+ 



19 



28 +9 
19 +17 



79 53 



34 

78 



25 +17 

10 - 4 

65 -13 



16 

42 

42 



25 

50 



+ 9 
-17 



|is total does not include ttie 327 grade HI pupils for whom no gain scores were available 
I ttie absence of a pre-test. 
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Standardized Test Evaluation of Projoct READ During School Year 1968-1969 

ill, TOE- .Wffi POST-TEST STATUS .MID PUPEj 3CCEE CHANOl PCR PROIECT READ PARTICIPANTS 
m SIX ELSdEJJTARY OTADE SEQUEUCESs STANPCED iffiADINU TEST, PARAGRAPH MEAITOTG 



Grade Seouenee 
Test Month & Year 
Test Level & Form 
(A.G.P, At Tasting] 
Pre-Teat Post-Test 

m. 

Ifay '69 

P I W 
(1.8) 


Elap- 

sed 

Time 

in 

Years 


Nuiriber 0: 
Pu- Seh- 
plla oolg 

327 18 


r| Per 
■ y cen- 

}| tile 


(A) 

Statue of Group! 
Grade Equivalent 
Pre- Post- 
Test Test Dlff 


, (B) 

1 Pupil 
; Gains : 
I 0 Mon. 

for-Mon 


I Seale 
1 in He 
4 AaG.P 
■f (Years 


1 

1 Per Cent 

I of Pupils 

1 Pre— Post 
) Test Test Dii 


75* 

50* 

25» 


1.7 

1.5 

1.5 




AGP/+ 

-1.0/h 


17 

n 83 

0 


m. IE 

May’68 May'69 

P I W P II W 

(1.8) (2.8) 


1.0 


35^ 19 


75* 

50«i 

25th 


1.6 2.9 +1.3 

1,5 2.0 +0.5 

1.4 1.7 +0.3 


29^ 


AGP/+ 

-.1/.9 

-1.0/+ 


12 27 +15 

ij 88 34 -54 

1 0 39 +39 


K2 H3 

May'68 ifay'69 

P IX W P II X 

(2.8) (3.8) 


1.0 


1 359 18 


75* 

50^» 

25* 


2.6 3.8 +1.2 

2.0 2.9 +0.9 

1.7 2.3 +0.6 


4o^ 


AGP/+ 

1 -.1/.9 


21 26 + 5 

43 25 _18 

1 36 49 +13 


10 m 

May *68 Jfey'69 
P II X Int.IX 
(3*8) (4.8) 


1.0 


1 108 6 


75* 

50« 

25* 


3.2 4.8 +1.6 

2.6 3.9 +1.3 

1-9 2.9 +1.0 


51 % 


1 -1.0/+I 
1 ACtP/+ 

-.I/.9 

-1.0/+ 


14 27 +13 ■ 

28 26 -2 

56 47 -11 


l 4 L5 

May’68 May'69 

P II W Int.DC 

(^•3) (3.3) 


1.0 


43 4 


75* 

50^ 

25th 


3.1 4.6 +1.5 

2.5 3.7 +1.2 

2.0 2.7 +0.7 


k 7 % 

72^ I 


AGP/+ 

-•I/.9 

-1.0/+ 


9 7 - 2 

9 21 +12 

82 72 -10 


h 4 H5 

May ' 68 May ' 69 

P II Y Int.IX 

(^.8) (5.8) 


1.0 


57 3 


75* 

50® 

25* 


4,0 6.0 +2.0 

3.2- 4.6 +1.4 

2.7 3.7 +1.0 




AGP/+ ' 
-.1/.9 

-1.0/+ 


16 26 +10 

16 16 0 

68 58 -10 


H6 

Oct *68 June '69 
Int.IIIir Int.IlY 
(6.1) (6,9) 


0.8 


375 9 


75* 

50® 

25* 


5.4 6.6 +1.2 

4.2 5.0 +0.8 

3*4 4.2 +0,8 


56^ [ 


AGF/+ 

-.I/.9 

-1.0/+ 


15 21+6 ■ 

13 12 1 ^ 

72 67-5 : 


Six Grade 
Sequences 

*Thia toi^l nn-t*. 


1296* 

27 




45^ 


AGP/+ 

-.1/.9 

-1.0/+ 


16 24+8 ' 

43 23 -20 : 

4 l 53 +12 t 



in the absence of a ^e«test. 



no gain scores were available 
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THREE-YEAR CO^ffARISON OF READING READINESS STAIUS AT END OF KINDERGARTEN, 
FOR PROJECT READ SCHOOLS AND ALL DISTRICT SCHOOLS 



1967 - Precedes Introduction of Project Read 

1968 - Includes Eight-Week Pilot of Project Read, March-May 

1969 - Includes First Complete Year of Project Read 



Metropolitan Readiness Test, Form A, Given in Late May or Early September 
(Test Score Percentile Equivalents Based on Publisher’s Standardization Norms 
on Which the "Typical" Pupil Ranked at SOth Percentile) 



PROJECT READ SCHOOLS 
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APPENDIX D 



LIST OF PRODUCTS AND DEVELOPERS 



The following is a list of products for which Product Development Reports 



will be prepared. 

1, Aritluaetic Proficiency Training Prograjn (APTP) 
Developeri Science Research Associates 

2, CLG Drug Education ProgrMi 
Developer I Creative Learning Group 

Cambridge , Massachusetts 

3, Cluster Concept Program 

Developer: Dr* Donald Maley and Dr* Walter Mietus 

University of Marylana 

4* Developmental Economic Education Program (DEEP) 
Developer: Joint Council on Economic Education 

5* DISTAR 

Developer: Siegfried Engelmann & Associates 

6. Facilitating Inquiry in the Classroom 

Developer: Northwest Regional Educational 

Laboratory 

7. First Year Coimnunieation Skills Program 

Developer: Southwest E_egional Laboratory for 

Educational Research ^ Development 

8. Frostlg Perceptual^Motor Skills Program 

Developer: Dr* Marianne Frostig 

9* Hawaii English Program 

Developer : Hawaii State Department of Ediication 

and the University of Hawaii 

10. Holt Social Studies Curriculum 
Developer : Dr , Edwin Fenton 



11. Individually Prescribed Ins true tion^Math 

Developer: Learning Research and Development Canter* 

University of Pittsburgh 

12 « Intermediate Science Currlculim Study 
Developer : Florida State University 

Dr. Ernest Burkman 

13. MATGH^ — ^Materials and Activities for Teachers and Children 
Davalopar : The Children’s Museim 

Boston, Massachusetts 



Carnegie Education Center 
Carnegie— Mellon University 
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14. Project PLAN 

Developer: Dr* John C. Flanagan and the 

American Institutes for Research 

15. Science: A Process Approach 

Developer: ^erican Association for the Advancement 

of Science j Commission on Science Education 

16. Scianea Curriculum Improvement Study 

Developer: Dr. Robert Karplus, Director 

University of Californlaj Berkeley 

17. Sesame Street 

Developer: Children’s Television Workshop 

18. Sullivan Reading Program 

Developer: Dr. M, L* Sullivan 

19. Taba Curriculum Development Project 

Developer: San Francisco State College 

20. Talking Typewriter 

Developer: Omar K. Moore and Responsive 

Environments Corporation 

21. Variable Modular Seheduling 

Developer: Stanford University and 

Educational Coordinates 
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